Golang CGO with large char pointer - SEGSERV - c++

I have a large amount of data being read from TagLib library and passed to GoLang (mpeg image data).
Here is where data is fetched:
void audiotags_mpeg_artwork(TagLib::MPEG::File *mpegFile, int id) {
TagLib::ID3v2::Tag *id3v2 = mpegFile->ID3v2Tag(false);
if (id3v2!=nullptr) {
const TagLib::ID3v2::FrameList frameList = id3v2->frameListMap()["APIC"];
for(auto it = frameList.begin(); it != frameList.end(); it++) {
TagLib::ID3v2::AttachedPictureFrame * frame = (TagLib::ID3v2::AttachedPictureFrame *)(*it);
if (frame!=nullptr && frame->size() > 0) {
const auto &apicBase64 = frame->picture().toBase64();
auto len = apicBase64.size();
if (len > 0) {
// Generate memory for key
char* key = new char[5];
memcpy(key, "APIC", 4);
key[4]='\0';
// Generate memory for picture data
char* val = new char[len];
memcpy (val, apicBase64.data(), len);
// Send to GoLang
go_map_audiotags(id, key, val);
// Free memory
delete[] key;
delete[] val;
}
}
}
}
}
At this point, go_map_autotags works (I use a similar method for other data). This also works for other picture data, however depending on the size this will crash with:
unexpected fault address 0x766a000
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x766a000 pc=0x404530b]
Within GoLang, I have the following export:
//export go_map_audiotags
func go_map_audiotags(id C.int, key *C.char, val *C.char) {
m := maps[int(id)]
k := strings.ToLower(C.GoString(key))
log.Println("go_map_audiotags k:", k) // <--- works
v := C.GoString(val) // <--- crashes
log.Println("go_map_audiotags v:", v) // <--- Does not reach
m[k] = v
}
Is there a bette way I should be transporting this data? I assume what's happening is:
1) The C.char limit be being reached
2) C++ is, for some reason, recycling the memory before setting v in GoLang

The data stored in val is not null-terminated. In your C code, when you make a copy using memcpy, the null terminator is not included. In the C code, change the code to:
// Generate memory for picture data
char* val = new char[len+1];
memcpy (val, apicBase64.data(), len);
val[len] = '\0';

Related

Intel OneAPI Video decoding memory leak when using C++ CLI

I am trying to use Intel OneAPI/OneVPL to decode a stream I receive from an RTSP Camera in C#. But when I run the code I get an enormous memory leak. Around 1-200MB per run, which is around once every second.
When I've collected a GoP from the camera where I know the first data is a keyframe I pass it as a byte array to my CLI and C++ code.
Here I expect it to decode all the frames and return decoded images. It receives 30 frames and returns 16 decoded images but has a memory leak.
I've tried to use Visual Studio memory profiler and all I can tell from it is that its unmanaged memory that's my problem. I've tried to override the "new" and "delete" method inside videoHandler.cpp to track and compare all allocations and deallocations and as far as I can tell everything is handled correctly in there. I cannot see any classes that get instantiated that do not get cleaned up. I think my issue is in the CLI class videoHandlerWrapper.cpp. Am I missing something obvious?
videoHandlerWrapper.cpp
array<imgFrameWrapper^>^ videoHandlerWrapper::decode(array<System::Byte>^ byteArray)
{
array<imgFrameWrapper^>^ returnFrames = gcnew array<imgFrameWrapper^>(30);
{
std::vector<imgFrame> frames(30); //Output from decoding process. imgFrame implements a deconstructor that will rid the data when exiting scope
std::vector<unsigned char> bytes(byteArray->Length); //Input for decoding process
Marshal::Copy(byteArray, 0, IntPtr((unsigned char*)(&((bytes)[0]))), byteArray->Length); //Copy from managed (C#) to unmanaged (C++)
int status = _pVideoHandler->decode(bytes, frames); //Decode
for (size_t i = 0; i < frames.size(); i++)
{
if (frames[i].size > 0)
returnFrames[i] = gcnew imgFrameWrapper(frames[i].size, frames[i].bytes);
}
}
//PrintMemoryUsage();
return returnFrames;
}
videoHandler.cpp
#define BITSTREAM_BUFFER_SIZE 2000000 //TODO Maybe higher or lower bitstream buffer. Thorough testing has been done at 2000000
int videoHandler::decode(std::vector<unsigned char> bytes, std::vector<imgFrame> &frameData)
{
int result = -1;
bool isStillGoing = true;
mfxBitstream bitstream = { 0 };
mfxSession session = NULL;
mfxStatus sts = MFX_ERR_NONE;
mfxSurfaceArray* outSurfaces = nullptr;
mfxU32 framenum = 0;
mfxU32 numVPPCh = 0;
mfxVideoChannelParam* mfxVPPChParams = nullptr;
void* accelHandle = NULL;
mfxVideoParam mfxDecParams = {};
mfxVersion version = { 0, 1 };
//variables used only in 2.x version
mfxConfig cfg = NULL;
mfxLoader loader = NULL;
mfxVariant inCodec = {};
std::vector<mfxU8> input_buffer;
// Initialize VPL session for any implementation of HEVC/H265 decode
loader = MFXLoad();
VERIFY(NULL != loader, "MFXLoad failed -- is implementation in path?");
cfg = MFXCreateConfig(loader);
VERIFY(NULL != cfg, "MFXCreateConfig failed")
inCodec.Type = MFX_VARIANT_TYPE_U32;
inCodec.Data.U32 = MFX_CODEC_AVC;
sts = MFXSetConfigFilterProperty(
cfg,
(mfxU8*)"mfxImplDescription.mfxDecoderDescription.decoder.CodecID",
inCodec);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for decoder CodecID");
sts = MFXCreateSession(loader, 0, &session);
VERIFY(MFX_ERR_NONE == sts, "Not able to create VPL session");
// Print info about implementation loaded
version = ShowImplInfo(session);
//VERIFY(version.Major > 1, "Sample requires 2.x API implementation, exiting");
if (version.Major == 1) {
mfxVariant ImplValueSW;
ImplValueSW.Type = MFX_VARIANT_TYPE_U32;
ImplValueSW.Data.U32 = MFX_IMPL_TYPE_SOFTWARE;
MFXSetConfigFilterProperty(cfg, (mfxU8*)"mfxImplDescription.Impl", ImplValueSW);
sts = MFXCreateSession(loader, 0, &session);
VERIFY(MFX_ERR_NONE == sts, "Not able to create VPL session");
}
// Convenience function to initialize available accelerator(s)
accelHandle = InitAcceleratorHandle(session);
bitstream.MaxLength = BITSTREAM_BUFFER_SIZE;
bitstream.Data = (mfxU8*)calloc(bytes.size(), sizeof(mfxU8));
VERIFY(bitstream.Data, "Not able to allocate input buffer");
bitstream.CodecId = MFX_CODEC_AVC;
std::copy(bytes.begin(), bytes.end(), bitstream.Data);
bitstream.DataLength = static_cast<mfxU32>(bytes.size());
memset(&mfxDecParams, 0, sizeof(mfxDecParams));
mfxDecParams.mfx.CodecId = MFX_CODEC_AVC;
mfxDecParams.IOPattern = MFX_IOPATTERN_OUT_SYSTEM_MEMORY;
sts = MFXVideoDECODE_DecodeHeader(session, &bitstream, &mfxDecParams);
VERIFY(MFX_ERR_NONE == sts, "Error decoding header\n");
numVPPCh = 1;
mfxVPPChParams = new mfxVideoChannelParam[numVPPCh];
for (mfxU32 i = 0; i < numVPPCh; i++) {
mfxVPPChParams[i] = {};
}
//mfxVPPChParams[0].VPP.FourCC = mfxDecParams.mfx.FrameInfo.FourCC;
mfxVPPChParams[0].VPP.FourCC = MFX_FOURCC_BGRA;
mfxVPPChParams[0].VPP.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
mfxVPPChParams[0].VPP.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
mfxVPPChParams[0].VPP.FrameRateExtN = 30;
mfxVPPChParams[0].VPP.FrameRateExtD = 1;
mfxVPPChParams[0].VPP.CropW = 1920;
mfxVPPChParams[0].VPP.CropH = 1080;
//Set value directly if input and output is the same.
mfxVPPChParams[0].VPP.Width = 1920;
mfxVPPChParams[0].VPP.Height = 1080;
//// USED TO RESIZE. IF INPUT IS THE SAME AS OUTPUT THIS WILL MAKE IT SHIFT A BIT. 1920x1080 becomes 1920x1088.
//mfxVPPChParams[0].VPP.Width = ALIGN16(mfxVPPChParams[0].VPP.CropW);
//mfxVPPChParams[0].VPP.Height = ALIGN16(mfxVPPChParams[0].VPP.CropH);
mfxVPPChParams[0].VPP.ChannelId = 1;
mfxVPPChParams[0].Protected = 0;
mfxVPPChParams[0].IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY | MFX_IOPATTERN_OUT_SYSTEM_MEMORY;
mfxVPPChParams[0].ExtParam = NULL;
mfxVPPChParams[0].NumExtParam = 0;
sts = MFXVideoDECODE_VPP_Init(session, &mfxDecParams, &mfxVPPChParams, numVPPCh); //This causes a MINOR memory leak!
outSurfaces = new mfxSurfaceArray;
while (isStillGoing == true) {
sts = MFXVideoDECODE_VPP_DecodeFrameAsync(session,
&bitstream,
NULL,
0,
&outSurfaces); //Big memory leak. 100MB pr run in the while loop.
switch (sts) {
case MFX_ERR_NONE:
// decode output
if (framenum >= 30)
{
isStillGoing = false;
break;
}
sts = WriteRawFrameToByte(outSurfaces->Surfaces[1], &frameData[framenum]);
VERIFY(MFX_ERR_NONE == sts, "Could not write 1st vpp output");
framenum++;
break;
case MFX_ERR_MORE_DATA:
// The function requires more bitstream at input before decoding can proceed
isStillGoing = false;
break;
case MFX_ERR_MORE_SURFACE:
// The function requires more frame surface at output before decoding can proceed.
// This applies to external memory allocations and should not be expected for
// a simple internal allocation case like this
break;
case MFX_ERR_DEVICE_LOST:
// For non-CPU implementations,
// Cleanup if device is lost
break;
case MFX_WRN_DEVICE_BUSY:
// For non-CPU implementations,
// Wait a few milliseconds then try again
break;
case MFX_WRN_VIDEO_PARAM_CHANGED:
// The decoder detected a new sequence header in the bitstream.
// Video parameters may have changed.
// In external memory allocation case, might need to reallocate the output surface
break;
case MFX_ERR_INCOMPATIBLE_VIDEO_PARAM:
// The function detected that video parameters provided by the application
// are incompatible with initialization parameters.
// The application should close the component and then reinitialize it
break;
case MFX_ERR_REALLOC_SURFACE:
// Bigger surface_work required. May be returned only if
// mfxInfoMFX::EnableReallocRequest was set to ON during initialization.
// This applies to external memory allocations and should not be expected for
// a simple internal allocation case like this
break;
default:
printf("unknown status %d\n", sts);
isStillGoing = false;
break;
}
}
sts = MFXVideoDECODE_VPP_Close(session); // Helps massively! Halves the memory leak speed. Closes internal structures and tables.
VERIFY(MFX_ERR_NONE == sts, "Error closing VPP session\n");
result = 0;
end:
printf("Decode and VPP processed %d frames\n", framenum);
// Clean up resources - It is recommended to close components first, before
// releasing allocated surfaces, since some surfaces may still be locked by
// internal resources.
if (mfxVPPChParams)
delete[] mfxVPPChParams;
if (outSurfaces)
delete outSurfaces;
if (bitstream.Data)
free(bitstream.Data);
if (accelHandle)
FreeAcceleratorHandle(accelHandle);
if (loader)
MFXUnload(loader);
return result;
}
imgFrameWrapper.h
public ref class imgFrameWrapper
{
private:
size_t size;
array<System::Byte>^ bytes;
public:
imgFrameWrapper(size_t u_size, unsigned char* u_bytes);
~imgFrameWrapper();
!imgFrameWrapper();
size_t get_size();
array<System::Byte>^ get_bytes();
};
imgFrameWrapper.cpp
imgFrameWrapper::imgFrameWrapper(size_t u_size, unsigned char* u_bytes)
{
size = u_size;
bytes = gcnew array<System::Byte>(size);
Marshal::Copy((IntPtr)u_bytes, bytes, 0, size);
}
imgFrameWrapper::~imgFrameWrapper()
{
}
imgFrameWrapper::!imgFrameWrapper()
{
}
size_t imgFrameWrapper::get_size()
{
return size;
}
array<System::Byte>^ imgFrameWrapper::get_bytes()
{
return bytes;
}
imgFrame.h
struct imgFrame
{
int size;
unsigned char* bytes;
~imgFrame()
{
if (bytes)
delete[] bytes;
}
};
MFXVideoDECODE_VPP_DecodeFrameAsync() function creates internal memory surfaces for the processing.
You should release surfaces.
Please check this link it's mentioning about it.
https://spec.oneapi.com/onevpl/latest/API_ref/VPL_structs_decode_vpp.html#_CPPv415mfxSurfaceArray
mfxStatus (*Release)(struct mfxSurfaceArray *surface_array)ΒΆ
Decrements the internal reference counter of the surface. (*Release) should be
called after using the (*AddRef) function to add a surface or when allocation
logic requires it.
And please check this sample.
https://github.com/oneapi-src/oneVPL/blob/master/examples/hello-decvpp/src/hello-decvpp.cpp
Especially, WriteRawFrame_InternalMem() function in https://github.com/oneapi-src/oneVPL/blob/17968d8d2299352f5a9e09388d24e81064c81c87/examples/util/util/util.h
It shows how to release surfaces.

c++ find and handle each lines is slower than php

Hello i created a program to handle a config file line by checking each lines and get the config blocks but for first time i made it with php and the speed was amazing. we have some blocks like this
Block {
}
php program can read each line and detect about 50,000 of this blocks in just 1 second after that i went to c++ to create my program in c++ but i saw a very very bad problem. my program was too slow (read 50,000 of this blocks in 55 seconds) while my php codes was exactly the same of c++ codes (in action and activity). php was 55x faster than c++ while the codes are the same.
this is my code in php
const PATH = "conf.txt";
if(!file_exists(PATH)) die("path_not_found");
if(!is_readable((PATH))) die("path_not_readable");
$Lines = explode("\r\n", file_get_contents(PATH));
class Block
{
public $Name;
public $Keys = array();
public $Blocks = array();
}
function Handle(& $Lines, $Start, & $Return_block, & $End_on)
{
for ($i = $Start; $i < count($Lines); $i++)
{
while (trim($Lines[$i]) != "")
{
$Pos1 = strpos($Lines[$i], "{");
$Pos2 = strpos($Lines[$i], "}");
if($Pos1 !== false && ($Pos2 === false || $Pos2 > $Pos1)) // Detect { in less position
{
$thisBlock = new Block();
$thisBlock->Name = trim(substr($Lines[$i], 0, $Pos1));
$Lines[$i] = substr($Lines[$i], $Pos1 + 1);
Handle($Lines, $i, $thisBlock, $i);
$Return_block->Blocks[] = $thisBlock;
}
else { // Detect } in less position than {
$Lines[$i] = substr($Lines[$i], $Pos2 + 1);
$End_on = $i;
return;
}
}
}
}
$DefaultBlock = new Block();
Handle($Lines, 0, $DefaultBlock, $NullValue);
$OutsideKeys = $DefaultBlock->Keys;
$Blocks = $DefaultBlock->Blocks;
echo "Found (".count($OutsideKeys).") keys and (".count($Blocks).") blocks.<br><br>";
and this is my code in C++
string Trim(string & s)
{
auto wsfront = std::find_if_not(s.begin(), s.end(), [](int c) {return std::isspace(c); });
auto wsback = std::find_if_not(s.rbegin(), s.rend(), [](int c) {return std::isspace(c); }).base();
return (wsback <= wsfront ? std::string() : std::string(wsfront, wsback));
}
class Block
{
private:
string Name;
vector <Block> Blocks;
public:
void Add(Block & thisBlock) { Blocks.push_back(thisBlock); }
Block(string Getname = string()) { Name = Getname; }
int Count() { return Blocks.size(); }
};
void Handle(vector <string> & Lines, size_t Start, Block & Return, size_t & LastPoint, bool CheckEnd = true)
{
for (size_t i = Start; i < Lines.size(); i++)
{
while (Trim(Lines[i]) != "")
{
size_t Pos1 = Lines[i].find("{");
size_t Pos2 = Lines[i].find("}");
if (Pos1 != string::npos && (Pos2 == string::npos || Pos1 < Pos2)) // Found {
{
string Name = Trim(Lines[i].substr(0, Pos1));
Block newBlock = Block(Name);
Lines[i] = Lines[i].substr(Pos1 + 1);
Handle(Lines, i, newBlock, i);
Return.Add(newBlock);
}
else { // Found }
Lines[i] = Lines[i].substr(Pos2 + 1);
return;
}
}
}
}
int main()
{
string Cont;
___PATH::GetFileContent("D:\\conf.txt", Cont);
vector <string> Lines = ___String::StringSplit(Cont, "\r\n");
Block Return;
size_t Temp;
// The problem (low handle speed) start from here not from including or split
Handle(Lines, 0, Return, Temp);
cout << "Is(" << Return.Count() << ")" << endl;
return 0;
}
as you can see, this codes are exactly the same in action but i don't know why php handling in this code is 55x faster than my c++ codes. you can create a txt file and create about 50,000 of this block's
Block {
}
and test it yourself. please help me to fix this. i am really confused (same codes but not same performance
php = 50,000 blocks and detect in 1 second
c++ = 50,000 blocks and detect in 55 seconds (and maybe more) !
i have no problem in my program design. because i got my performance completely on php but my problem is on c++ that is 55x slower than php in same code action !
i am using (visual studio 2017) to compile this program (c++)
First, "code" is singular, not plural.
C++ is a very different language than php. It is not "the same code", and it is nowhere near the same in action.
For example, these two lines:
Block newBlock = Block(Name);
Return.Add(newBlock);
First create a Block on the stack, and then call Block's copy constructor to make another one inside the vector. You then throw away the stack object.
Also, vectors guarantee that they are contiguous, so as you add new Blocks via your Add method, vector will occasionally stop, allocate another chunk of memory (twice as big as the last one, iirc), copy everything over to that new chunk, and then free the old one. Either preallocate the vector (via vector::reserve()), or consider using something like a deque that doesn't guarantee continuity in memory if you don't need that property.
I also don't know what ___String::StringSplit does, but you are almost certain to have the same vector growth problem in reading your file.
Culprit is in these 2 lines:
Handle(Lines, i, newBlock, i);
Return.Add(newBlock);
Let's say you have 5 levels of 1 block each. What Happens on bottom one? You copy one instance of block. What happens on level 4? You copy 2 blocks (parent and its child). So for level 5 you make 15 copies - 1+2+3+4+5. Look at this diagram:
Handle level1 copies 5 blocks (`Return`->level4->level3->level4->level5)
Handle level2 copies 4 blocks (`Return`->level3->level4->level5)
Handle level3 copies 3 blocks (`Return`->level4->level5
Handle level4 copies 2 blocks (`Return`->level5)
Handle level5 copies 1 block (`Return`)
Formula is:
S = ( N + N^2 ) / 2
so for levels 20 you would do 210 copies and so on.
Suggestion is to use move semantics to avoid this copy:
// change method Add to this
void Add(Block thisBlock) { Blocks.push_back(std::move(thisBlock)); }
// and change this call
Return.Add( std::move( newBlock ) );
Or allocate blocks dynamically using smart pointers
Out of simple curiousity, try this Trim implementation instead:
void _Trim(std::string& result, const std::string& s) {
const auto* ptr = s.data();
const auto* left = ptr;
const auto* end = s.data() + s.size();
while (ptr < end && std::isspace(*ptr)) {
++ptr;
}
if (ptr == end) {
result = "";
return;
}
left = ptr;
while (end > left && std::isspace(*(end-1))) {
--end;
}
result = std::string(left, end);
}
std::string Trim(const std::string& s) {
// Not sure if RVO would fire for direct implementation of _Trim here
std::string result;
_Trim(result, s);
return result;
}
And another optimization:
void Add(Block& thisBlock) {
Blocks.push_back(std::move(thisBlock));
}
// Don't use thisBlock after call to this function. It is
// far from being pretty but it should avoid *lots* of copies.
I wonder if you'll get better result. Pls let me know.

std::list copy to std::vector skipping elements

I've run across a rather bizarre exception while running C++ code in my objective-C application. I'm using libxml2 to read an XSD file. I then store the relevant tags as instances of the Tag class in an std::list. I then copy this list into an std::vector using an iterator on the list. However, every now and then some elements of the list aren't copied to the vector. Any help would be greatly appreciated.
printf("\n length list = %lu, length vector = %lu\n",XSDFile::tagsList.size(), XSDFile::tags.size() );
std::list<Tag>::iterator it = XSDFile::tagsList.begin();
//result: length list = 94, length vector = 0
/*
for(;it!=XSDFile::tagsList.end();++it)
{
XSDFile::tags.push_back(*it); //BAD_ACCESS code 1 . . very bizarre . . . . 25
}
*/
std::copy (XSDFile::tagsList.begin(), XSDFile::tagsList.end(), std::back_inserter (XSDFile::tags));
printf("\n Num tags in vector = %lu\n", XSDFile::tags.size());
if (XSDFile::tagsList.size() != XSDFile::tags.size())
{
printf("\n length list = %lu, length vector = %lu\n",XSDFile::tagsList.size(), XSDFile::tags.size() );
//result: length list = 94, length vector = 83
}
I've found the problem. The memory was corrupted causing the std::list to become corrupted during the parsing of the XSD. I parse the XSD using a function start_element.
xmlSAXHandler handler = {0};
handler.startElement = start_element;
I used malloc guard in xcode to locate the use of freed memory. It pointed to the line:
std::strcpy(message, (char*)name);
So I removed the malloc (actually commented in the code) and it worked. The std::vector now consistently copies all 94 entries of the list. If anyone has an explanation as to why this worked that would be great.
static void start_element(void * ctx, const xmlChar *name, const xmlChar **atts)
{
// int len = strlen((char*)name);
// char *message = (char*)malloc(len*sizeof(char));
// std::strcpy(message, (char*)name);
if (atts != NULL)
{
// atts[0] = type
// atts[1] = value
// len = strlen((char*)atts[1]);
// char *firstAttr = (char*)malloc(len*sizeof(char));
// std::strcpy(firstAttr, (char*)atts[1]);
if(strcmp((char*)name, "xs:include")==0)
{
XSDFile xsd;
xsd.ReadXSDTypes((char*)atts[1]);
}
else if(strcmp((char*)name, "xs:element")==0)
{
doElement(atts);
}
else if(strcmp((char*)name, "xs:sequence")==0)
{
//set the default values
XSDFile::sequenceMin = XSDFile::sequenceMax = 1;
if (sizeof(atts) == 4)
{
if(strcmp((char*)atts[3],"unbounded")==0)
XSDFile::sequenceMax = -1;
int i = 0;
while(atts[i] != NULL)
{
//atts[i] = name
//atts[i+i] = value
std::string name((char*)atts[i]);
std::string value((char*)atts[i+1]);
if(name=="minOccurs")
XSDFile::sequenceMin = (atoi(value.c_str()));
else if(name=="maxOccurs")
XSDFile::sequenceMax = (atoi(value.c_str()));
i += 2;
}
}
}
}
//free(message);
}

How to encrypt and decrypt a const char* in WinRT

I have been trying to write encrypt and decrypt functions whose signatures require the input and the output strings to be void* type only. The code works fine if the inputs can be specified as IBuffer^ but in the other case the source string and the encrypted->decrypted string do not match.
CodeIBuffer^ byteArrayToIBufferPtr(byte *source, int size)
{
Platform::ArrayReference<uint8> blobArray(source, size);
IBuffer ^buffer = CryptographicBuffer::CreateFromByteArray(blobArray);
return buffer;
}
byte* IBufferPtrToByteArray(IBuffer ^buffer)
{
Array<unsigned char,1U> ^platArray = ref new Array<unsigned char,1U>(256);
CryptographicBuffer::CopyToByteArray(buffer,&platArray);
byte *dest = platArray->Data;
return dest;
}
int DataEncryption::encryptData(EncryptionAlgorithm algo, int keySize, void* srcData, const unsigned int srcSize,
void*& encData, unsigned int& encSize)
{
LOG_D(TAG, "encryptData()");
if(srcData == nullptr)
{
LOG_E(TAG,"");
return DataEncryption::RESULT_EMPTY_DATA_ERROR;
}
if(srcSize == 0)
{
LOG_E(TAG,"");
return DataEncryption::RESULT_SIZE_ZERO_ERROR;
}
IBuffer^ encrypted;
IBuffer^ buffer;
IBuffer^ iv = nullptr;
String^ algName;
bool cbc = false;
switch (algo)
{
case DataEncryption::ENC_DEFAULT:
algName = "AES_CBC";
cbc = true;
break;
default:
break;
}
// Open the algorithm provider for the algorithm specified on input.
SymmetricKeyAlgorithmProvider^ Algorithm = SymmetricKeyAlgorithmProvider::OpenAlgorithm(algName);
// Generate a symmetric key.
IBuffer^ keymaterial = CryptographicBuffer::GenerateRandom((keySize + 7) / 8);
CryptographicKey^ key;
try
{
key = Algorithm->CreateSymmetricKey(keymaterial);
}
catch(InvalidArgumentException^ e)
{
LOG_E(TAG,"encryptData(): Could not create key.");
return DataEncryption::RESULT_ERROR;
}
// CBC mode needs Initialization vector, here just random data.
// IV property will be set on "Encrypted".
if (cbc)
iv = CryptographicBuffer::GenerateRandom(Algorithm->BlockLength);
// Set the data to encrypt.
IBuffer ^srcDataBuffer = byteArrayToIBufferPtr(static_cast<byte*>(srcData),256);
// Encrypt and create an authenticated tag.
encrypted = CryptographicEngine::Encrypt(key, srcDataBuffer, iv);
//encData = encrypted;
byte *bb = IBufferPtrToByteArray(encrypted);
encData = IBufferPtrToByteArray(encrypted);
encSize = encrypted->Length;
return DataEncryption::RESULT_SUCCESS;
}
int DataEncryption::decryptData(EncryptionAlgorithm algo, int keySize, void* encData, const unsigned int encSize,
void*& decData, unsigned int& decSize)
{
LOG_D(TAG, "decryptData()");
if(encData == nullptr)
{
LOG_E(TAG,"");
return DataEncryption::RESULT_EMPTY_DATA_ERROR;
}
if(encSize == 0)
{
LOG_E(TAG,"");
return DataEncryption::RESULT_SIZE_ZERO_ERROR;
}
IBuffer^ encrypted;
IBuffer^ decrypted;
IBuffer^ iv = nullptr;
String^ algName;
bool cbc = false;
switch (algo)
{
case DataEncryption::ENC_DEFAULT:
algName = "AES_CBC";
cbc = true;
break;
default:
break;
}
// Open the algorithm provider for the algorithm specified on input.
SymmetricKeyAlgorithmProvider^ Algorithm = SymmetricKeyAlgorithmProvider::OpenAlgorithm(algName);
// Generate a symmetric key.
IBuffer^ keymaterial = CryptographicBuffer::GenerateRandom((keySize + 7) / 8);
CryptographicKey^ key;
try
{
key = Algorithm->CreateSymmetricKey(keymaterial);
}
catch(InvalidArgumentException^ e)
{
LOG_E(TAG,"encryptData(): Could not create key.");
return DataEncryption::RESULT_ERROR;
}
// CBC mode needs Initialization vector, here just random data.
// IV property will be set on "Encrypted".
if (cbc)
iv = CryptographicBuffer::GenerateRandom(Algorithm->BlockLength);
// Set the data to decrypt.
byte *cc = static_cast<byte*>(encData);
IBuffer ^encDataBuffer = byteArrayToIBufferPtr(cc,256);
// Decrypt and verify the authenticated tag.
decrypted = CryptographicEngine::Decrypt(key, encDataBuffer, iv);
byte *bb = IBufferPtrToByteArray(decrypted);
decData = IBufferPtrToByteArray(decrypted);
decSize = decrypted->Length;
return DataEncryption::RESULT_SUCCESS;
}
I'm guessing that the problem is with this function:
byte* IBufferPtrToByteArray(IBuffer ^buffer)
{
Array<unsigned char,1U> ^platArray = ref new Array<unsigned char,1U>(256);
CryptographicBuffer::CopyToByteArray(buffer,&platArray);
byte *dest = platArray->Data;
return dest;
}
What you're doing there is allocating a new Platform::Array<byte>^ with 1 reference, then getting a pointer to its internally-managed storage, then returning that pointer-- at which point the Array is being dereferenced and is thus deallocating its underlying storage. Thus the pointer you return refers to freed memory. The next allocation is likely to overwrite those bytes.
What you'll need to do is take the return-by-reference Array<byte>^ from CopyToByteArray() (which creates a new Array, presumably wrapping the bytes of the input IBuffer^, and returns it) and copy that array's contents.
Your end result will function similarly to this snippet from the Readium SDK project, which takes a std::string instance, hashes it using SHA-1, and copies the hash data into a member variable uint8_t _key[KeySize]:
using namespace ::Platform;
using namespace ::Windows::Foundation::Cryptography;
using namespace ::Windows::Foundation::Cryptography::Core;
auto byteArray = ArrayReference<byte>(reinterpret_cast<byte*>(const_cast<char*>(str.data())), str.length());
auto inBuf = CryptographicBuffer::CreateFromByteArray(byteArray);
auto keyBuf = HashAlgorithmProvider::OpenAlgorithm(HashAlgorithmNames::Sha1)->HashData(inBuf);
Array<byte>^ outArray = nullptr;
CryptographicBuffer::CopyToByteArray(keyBuf, &outArray);
memcpy_s(_key, KeySize, outArray->Data, outArray->Length);
The steps:
Create an ArrayReference<byte> corresponding to the bytes in the std::string (no copying).
Pass that to CryptographicBuffer::CreateFromByteArray() to get your IBuffer^. Still no copying of data.
Call your hash/encryption function, passing the IBuffer^ you just made. You get another IBuffer^ in return, which may or may not be using the exact same storage (that's really up to the implementation of the algorithm, I think).
Create a variable of type Array<byte>^. Don't allocate an object, you're going to be given one by reference.
Pass the address of that object into CryptographicBuffer::CopyToByteArray() to receive a copy of your key data.
While that Array^ remains valid, copy its bytes into your native array.

Function has corrupt return value

I have a situation in Visual C++ 2008 that I have not seen before. I have a class with 4 STL objects (list and vector to be precise) and integers.
It has a method:
inline int id() { return m_id; }
The return value from this method is corrupt, and I have no idea why.
debugger screenshot http://img687.imageshack.us/img687/6728/returnvalue.png
I'd like to believe its a stack smash, but as far as I know, I have no buffer over-runs or allocation issues.
Some more observations
Here's something that puts me off. The debugger prints right values in the place mentioned // wrong ID.
m_header = new DnsHeader();
assert(_CrtCheckMemory());
if (m_header->init(bytes, size))
{
eprintf("0The header ID is %d\n", m_header->id()); // wrong ID!!!
inside m_header->init()
m_qdcount = ntohs(h->qdcount);
m_ancount = ntohs(h->ancount);
m_nscount = ntohs(h->nscount);
m_arcount = ntohs(h->arcount);
eprintf("The details are %d,%d,%d,%d\n", m_qdcount, m_ancount, m_nscount, m_arcount);
// copy the flags
// this doesn't work with a bitfield struct :(
// memcpy(&m_flags, bytes + 2, sizeof(m_flags));
//unpack_flags(bytes + 2); //TODO
m_init = true;
}
eprintf("Assigning an id of %d\n", m_id); // Correct ID.
return
m_header->id() is an inline function in the header file
inline int id() { return m_id; }
I don't really know how best to post the code snippets I have , but here's my best shot at it. Please do let me know if they are insufficient:
Class DnsHeader has an object m_header inside DnsPacket.
Main body:
DnsPacket *p ;
p = new DnsPacket(r);
assert (_CrtCheckMemory());
p->add_bytes(buf, r); // add bytes to a vector m_bytes inside DnsPacket
if (p->parse())
{
read_packet(sin, *p);
}
p->parse:
size_t size = m_bytes.size(); // m_bytes is a vector
unsigned char *bytes = new u_char[m_bytes.size()];
copy(m_bytes.begin(), m_bytes.end(), bytes);
m_header = new DnsHeader();
eprintf("m_header allocated at %x\n", m_header);
assert(_CrtCheckMemory());
if (m_header->init(bytes, size)) // just set the ID and a bunch of other ints here.
{
size_t pos = DnsHeader::SIZE; // const int
if (pos != size)
; // XXX perhaps generate a warning about extraneous data?
if (ok)
m_parsed = true;
}
else
{
m_parsed = false;
}
if (!ok) {
m_parsed = false;
}
return m_parsed;
}
read_packet:
DnsHeader& h = p.header();
eprintf("The header ID is %d\n", h.id()); // ID is wrong here
...
DnsHeader constructor:
m_id = -1;
m_qdcount = m_ancount = m_nscount = m_arcount = 0;
memset(&m_flags, 0, sizeof(m_flags)); // m_flags is a struct
m_flags.rd = 1;
p.header():
return *m_header;
m_header->init: (u_char* bytes, int size)
header_fmt *h = (header_fmt *)bytes;
m_id = ntohs(h->id);
eprintf("Assigning an id of %d/%d\n", ntohs(h->id), m_id); // ID is correct here
m_qdcount = ntohs(h->qdcount);
m_ancount = ntohs(h->ancount);
m_nscount = ntohs(h->nscount);
m_arcount = ntohs(h->arcount);
You seem to be using a pointer to an invalid class somehow. The return value shown is the value that VS usually uses to initialize memory with:
2^32 - 842150451 = 0xCDCDCDCD
You probably have not initialized the class that this function is a member of.
Without seeing more of the code in context.. it might be that the m_id is out of the scope you expect it to be in.
Reinstalled VC++. That fixed everything.
Thank you for your time and support everybody! :) Appreciate it!