I am trying to create code that places half a dozen variables into a char* and they are almost all multi-byte variables. Since they are all going into the same char* I'd like to be able to copy the data into the char* as easily as possible.
If there is no "standard" way for me to do this, I can easily make an inline function that deals with this for me. I was, however, hoping there would be a simple way of doing this, perhaps like:
memcpy(char* target, char* source, int target_start_position, int source_length)
So to elaborate on that:
I'd be copying all the data from source, into target. I'd copy the data to target[target_start_position] Then source_length would merely be for the function to know how much data it is going to be reading from the source.
What I actually tried. Perhaps you can see what my thought process was however.
=================
This is as simple as I was hoping to get things:
(Which I realize doesn't work because I am assigning pointers as values)
char* data;
unsigned __int64 pos = 0;
int string_len = sName.size();
unsigned __int64 data_size = sizeof(string_len) + string_len + (sizeof(int) * 10) + 3;
data = new char[data_size];
data[pos] = (char const*)(&string_len);
pos += sizeof(string_len);
data[pos] = (char const*)(&sName);
pos += string_len;
data[pos] = (char const*)(&iStatPoints);
pos += sizeof(int);
Note: I apologize for the title, that was the most precise way of describing the issue I could think of.
you can get the desired affect with pointer arithmetic. For example your desired function could be implemented as follows:
void *memcpy2(void *dest, const void *src, size_t offset, size_t length) {
return memcpy((char*)dest + offset, src, length);
}
Related
I have an array of chars, stored in a char*. From this pointer I want to copy the elements from a specific offset to another to a std::vector<uint16_t>. I need to combine always two chars, cause they are meant to be one 16 bit value.
The start and end offset is known and I know this values should be integers. But what is the best way to cast and copy this array. It's about 25000 uint values, so it should be an efficient way.
Following a code snippet to make it more clear. The problem with this snippet, it uses each char value as an own integer, but should combine always two chars two one 16 bit value. So the resulting vector has a size of 50688, but it should be 25344.
Simple:
char * buf = new char[bufSize];
std::vector<uint16_t> vec;
vec.reserve(numPix);
vec.insert(vec.begin(), &buf[startOffset], &buf[endOffset]);
Original:
const size_t numPixel = m_cameraParams.width * m_cameraParams.height; // 25344
const size_t numBytesDistance = numPixel * m_distanceByteDepth; // 50688
const size_t numBytesIntensity = numPixel * m_intensityByteDepth; // 50688
const size_t numBytesConfidence = numPixel * m_confidenceByteDepth; // 50688
std::vector<uint16_t> m_distanceMap;
std::vector<uint16_t> m_intensityMap;
std::vector<uint16_t> m_confidenceMap;
char * inputBuffer = new char[bufferSize];
// buffer filled with data received from tcp connection
//-----------------------------------------------
// Extract the Images depending on the informations extracted from the XML part
size_t offsetDistance = offset;
size_t offsetIntensity = offsetDistance + numBytesDistance;
size_t offsetConfidence = offsetIntensity + numBytesIntensity;
offset = offsetConfidence + numBytesConfidence;
m_distanceMap.reserve(numPixel);
m_distanceMap.insert(m_distanceMap.begin(), &inputBuffer[offsetDistance], &inputBuffer[offsetIntensity]);
m_intensityMap.reserve(numPixel);
m_intensityMap.insert(m_intensityMap.begin(), &inputBuffer[offsetIntensity], &inputBuffer[offsetConfidence]);
m_confidenceMap.reserve(numPixel);
m_confidenceMap.insert(m_confidenceMap.begin(), &inputBuffer[offsetConfidence], &inputBuffer[offset]);
I have attached my code below. I do not see what I am doing wrong. I have a struct that I am trying to serialize into a byte array. I have wrote some some simple code to test it. It all appears to work during runtime when I print out the values of objects, but once I hit return 0 it throws the error:
Run-Time Check Failure #2 - Stack around the variable 'command' was corrupted.
I do not see the issue. I appreciate all help.
namespace CommIO
{
enum Direction {READ, WRITE};
struct CommCommand
{
int command;
Direction dir;
int rwSize;
BYTE* wData;
CommCommand(BYTE* bytes)
{
int offset = 0;
int intsize = sizeof(int);
command = 0;
dir = READ;
rwSize = 0;
memcpy(&command, bytes + offset, intsize);
offset += intsize;
memcpy(&dir, bytes + offset, intsize);
offset += intsize;
memcpy(&rwSize, bytes + offset, intsize);
offset += intsize;
wData = new BYTE[rwSize];
if (dir == WRITE)
{
memcpy(&wData, bytes + offset, rwSize);
}
}
CommCommand() {}
}
int main()
{
CommIO::CommCommand command;
command.command = 0x6AEA6BEB;
command.dir = CommIO::WRITE;
command.rwSize = 128;
command.wData = new BYTE[command.rwSize];
for (int i = 0; i < command.rwSize; i++)
{
command.wData[i] = i;
}
command.print();
CommIO::CommCommand command2(reinterpret_cast<BYTE*>(&command));
command2.print();
cin.get();
return 0;
}
The following points mentioned in comments are most likely the causes of your problem.
You seem to be assuming that the size of Direction is the same as the size of an int. That may indeed be the case, but C++ does not guarantee it.
You also seem to be assuming that the members of CommIO::CommCommand will be laid out in memory without any padding between, which again may happen to be the case, but is not guaranteed.
There are couple of ways to fix the that.
Make sure that you fill up the BYTE array in the calling function with matching objects, or
Simply cast the BYTE* to CommCommand* and access the members directly.
For (1), you can use:
int command = 0x6AEA6BEB;
int dir = CommIO::WRITE;
int rwSize = 128;
totatlSize = rwSize + 3*sizeof(int);
BYTE* data = new BYTE[totalSize];
int offset = 0;
memcpy(data + offset, &comand, sizeof(int));
offset += sizeof(int);
memcpy(data + offset, &dir, sizeof(int));
offset += sizeof(int);
memcpy(data + offset, &rwSize, sizeof(int));
offset += sizeof(int);
for (int i = 0; i < rwSize; i++)
{
data[i + offset] = i;
}
CommIO::CommCommand command2(data);
For (2), you can use:
CommCommand(BYTE* bytes)
{
CommCommand* in = reinterpret_cast<CommCommand*>(bytes);
command = in->command;
dir = in->dir;
rwSize = in->size;
wData = new BYTE[rwSize];
if (dir == WRITE)
{
memcpy(wData, in->wData, rwSize);
}
}
The other error is that you are using
memcpy(&wData, bytes + offset, rwSize);
That is incorrect since you are treating the address of the variable as though it can hold the data. It cannot.
You need to use:
memcpy(wData, bytes + offset, rwSize);
The memory for your struct is laid out without padding, this can be rectified by adding the macro #pragma pack(1) at the start of the struct and #pragma pop() at the end of the struct - check its syntax though.
For your struct to byte conversion, I would use something simple as:
template<typename T, typename IteratorForBytes>
void ConvertToBytes(const T& t, IteratorForBytes bytes, std::size_t pos = 0)
{
std::advance(bytes, pos);
const std::size_t length = sizeof(t);
const uint8_t* temp = reinterpret_cast<const uint8_t*>(&t);
for (std::size_t i = 0; i < length; ++i)
{
(*bytes) = (*temp);
++temp;
++bytes;
}
}
Where T is the is the struct in your case your Command struct and bytes would be the array.
CommIO::CommCommand command;
command.wData = new BYTE[command.rwSize];
ConvertToBytes(command, command.wData);
The resulting array would contain the expected bytes You could specify the offset as well as an extra parameter if you want to start filling your byte array from a particular location
The main problem is here:
memcpy(&wData, bytes + offset, rwSize);
Member wData is a BYTE *, and you seem to mean to copy bytes into the space to which it points. Instead, you are copying data into the memory where the pointer value itself is stored. Therefore, if you copy more bytes than the size of the pointer then you will overrun its bounds and produce undefined behavior. In any case, you are trashing the original pointer value. You probably want this, instead:
memcpy(wData, bytes + offset, rwSize);
Additionally, although the rest of the deserialization code may be right for your actual serialization format, it is not safe to assume that it is right for the byte sequence you present to it in your test program via
CommIO::CommCommand command2(reinterpret_cast<BYTE*>(&command));
As detailed in comments, you are making assumptions about the layout in memory of a CommIO::CommCommand that C++ does not guarantee will hold.
At
memcpy(&wData, bytes + offset, rwSize);
you copy from the location of the wData pointer and to the location of the wData pointer of the new CommCommand. But you want to copy from and to the location that the pointer points to. You need to dereference. You corrupt the heap, because you have only sizeof(BYTE*) space (plus some extra, because heap blocks cannot be arbitrarily small), but you copy rwSize bytes, which is 128 bytes. What you probably meant to write is:
memcpy(wData, *(BYTE*)(bytes + offset), rwSize);
which would take use the pointer stored at bytes + offset, rather than the value of bytes + offset itself.
You also assume that your struct is tightly packed. However, C++ does not guarantee that. Is there a reason why you do not override the default copy constructor rather than write this function?
What are the right equivalent of unsigned char or unsigned char* in go? Or am I even doing this right?
I have this C++ class:
class ArcfourPRNG
{
public:
ArcfourPRNG();
void SetKey(unsigned char *pucKeyData, int iKeyLen);
void Reset();
unsigned char Rand();
private:
bool m_bInit;
unsigned char m_aucState0[256];
unsigned char m_aucState[256];
unsigned char m_ucI;
unsigned char m_ucJ;
unsigned char* m_pucState1;
unsigned char* m_pucState2;
unsigned char m_ucTemp;
};
I am trying to rewrite it to go:
type ArcfourPRNG struct {
m_bInit bool
m_aucState0 [256]byte
m_aucState [256]byte
m_ucI, m_ucJ []byte
*m_pucState1 []byte
*m_pucState2 []byte
m_ucTemp []byte
}
func (arc4 *ArcfourPRNG) SetKey(pucKeyData []byte, iKeyLen int) {
func (arc4 *ArcfourPRNG) Reset() {
func (arc4 *ArcfourPRNG) Rand() uint {
Well, I just started with go a few hours ago. So this is still confusing me.
A function
for(i=0; i<256; i++)
{
m_pucState1 = m_aucState0 + i;
m_ucJ += *m_pucState1 + *(pucKeyData+m_ucI);
m_pucState2 = m_aucState0 + m_ucJ;
//Swaping
m_ucTemp = *m_pucState1;
*m_pucState1 = *m_pucState2;
*m_pucState2 = m_ucTemp;
m_ucI = (m_ucI + 1) % iKeyLen;
}
memcpy(m_aucState, m_aucState0, 256); // copy(aucState[:], aucState0) ?
Hopefully this can clear a few things up for you.
For storing raw sequences of bytes, use a slice []byte. If you know exactly how long the sequence will be, you can specify that, e.g. [256]byte but you cannot resize it later.
While Go has pointers, it does not have pointer arithmetic. So you will need to use integers to index into your slices of bytes.
For storing single bytes, byte is sufficient; you don't want a slice of bytes. Where there are pointers in the C++ code used to point to specific locations in the array, you'll simply have an integer index value that selects one element of a slice.
Go strings are not simply sequences of bytes, they are sequences of UTF-8 characters stored internally as runes, which may have different lengths. So don't try to use strings for this algorithm.
To reimplement the algorithm shown, you do not need either pointers or pointer arithmetic at all. Instead of keeping pointers into the byte arrays as you would in C++, you'll use int indexes into the slices.
This is kind of hard to follow since it's virtually all pointer arithmetic. I would want to have a description of the algorithm handy while converting this (and since this is probably a well-known algorithm, that should not be hard to find). I'm not going to do the entire conversion for you, but I'll demonstrate with hopefully a simpler example. This prints each character of a string on a separate line.
C++:
unsigned char *data = "Hello World";
unsigned char *ptr = 0;
for (int i = 0; i < std::strlen(data); i++) {
ptr = i + data;
std::cout << *ptr << std::endl;
}
Go:
data := []byte("Hello World")
for i := 0; i < len(data); i++ {
// The pointer is redundant already
fmt.Println(data[i:i+1])
}
So, learn about Go slices, and when you do reimplement this algorithm you will likely find the code to be somewhat simpler, or at least easier to understand, than its C++ counterpart.
I want to assign string to char array
here is code -
char *resultArray[100];
int cnt = 0;
void MySAX2Handler::startElement(const XMLCh* const uri, const XMLCh* const localname,
const XMLCh* const qname, const Attributes& attrs)
{
char* message = XMLString::transcode(localname);
resultArray[cnt] = message;
cnt++;
for (int idx = 0; idx < attrs.getLength(); idx++)
{
char* attrName = XMLString::transcode(attrs.getLocalName(idx));
char* attrValue = XMLString::transcode(attrs.getValue(idx));
resultArray[cnt] = attrName;
cnt++;
resultArray[cnt] = attrValue;
cnt++;
}
XMLString::release(&message);
}
after iterating through resultArray, its printing some garbage values
attrName and attrValue are both pointers and char is a type of integer, so the assignment operator will simply copy the value of the pointer (the address), not the content.
Either loop through the strings or use some version of strcpy(), or indeed one of the C++ string libraries.
First off, when you declare your array, you should use
char *resultArray = new ResultArray[100];
vs
char *resultArray[100];
Next, you shouldn't ever use a loop to fill up a bounded array without knowing the limits of the loop. You will overflow your array, and cause a segfault. Unless of course you know your length won't reach close to 100. (still a bad idea, but to slap something together quickly, I won't complain)
Without more information, I can't tell you why you're getting garbage. How are you printing these out? What do those 'transcode' functions do? Where is the data coming from?
PS - remember you can use
resultArray[cnt++] = attrName;
resultArray[cnt++] = attrValue;
I need an array of this struct allocated in one solid chunk of memory. The length of "char *extension" and "char *type" are not known at compile time.
struct MIMETYPE
{
char *extension;
char *type;
};
If I used the "new" operator to initialize each element by itself, the memory may be scattered. This is how I tried to allocate a single contiguous block of memory for it:
//numTypes = total elements of array
//maxExtension and maxType are the needed lengths for the (char*) in the struct
//std::string ext, type;
unsigned int size = (maxExtension+1 + maxType+1) * numTypes;
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size);
But, when I try to load the data in like this, the data is all out of order and scattered when I try to access it later.
for(unsigned int i = 0; i < numTypes; i++)
{
//get data from file
getline(fin, line);
stringstream parser.str(line);
parser >> ext >> type;
//point the pointers at a spot in the memory that I allocated
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension);
//copy the data into the elements
strcpy(mimeTypes[i].extension, ext.c_str());
strcpy(mimeTypes[i].type, type.c_str());
}
can anyone help me out?
EDIT:
unsigned int size = (maxExtension+1 + maxType+1);
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size * numTypes);
for(unsigned int i = 0; i < numTypes; i++)
strcpy((char*)(mimeTypes + (i*size)), ext.c_str());
strcpy((char*)(mimeTypes + (i*size) + (maxExtension+1)), type.c_str());
You mix 2 allocation:
1) manage array of MIMETYPE and
2) manage array of characters
May be (I don't really understand your objectives):
struct MIMETYPE
{
char extension[const_ofmaxExtension];
char type[maxType];
};
would be better to allocate linear items in form:
new MIMETYPE[numTypes];
I'll put aside the point that this is premature optimization (and that you ought to just use std::string, std::vector, etc), since others have already stated that.
The fundamental problem I'm seeing is that you're using the same memory for both the MIMETYPE structs and the strings that they'll point to. No matter how you allocate it, a pointer itself and the data it points to cannot occupy the exact same place in memory.
Lets say you needed an array of 3 types and had MIMETYPE* mimeTypes pointing to the memory you allocated for them.
That means you're treating that memory as if it contains:
8 bytes: mime type 0
8 bytes: mime type 1
8 bytes: mime type 2
Now, consider what you're doing in this next line of code:
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
extension is being set to point to the same location in memory as the MIMETYPE struct itself. That is not going to work. When subsequent code writes to the location that extension points to, it overwrites the MIMETYPE structs.
Similarly, this code:
strcpy((char*)(mimeTypes + (i*size)), ext.c_str());
is writing the string data in the same memory that you otherwise want to MIMETYPE structs to occupy.
If you really want store all the necessary memory in one contiguous space, then doing so is a bit more complicated. You would need to allocate a block of memory to contain the MIMETYPE array at the start of it, and then the string data afterwards.
As an example, lets say you need 3 types. Lets also say the max length for an extension string (maxExtension) is 3 and the max length for a type string (maxType) is 10. In this case, your block of memory needs to be laid out as:
8 bytes: mime type 0
8 bytes: mime type 1
8 bytes: mime type 2
4 bytes: extension string 0
11 bytes: type string 0
4 bytes: extension string 1
11 bytes: type string 1
4 bytes: extension string 2
11 bytes: type string 2
So to allocate, setup, and fill it all correctly you would want to do something like:
unsigned int mimeTypeStringsSize = (maxExtension+1 + maxType+1);
unsigned int totalSize = (sizeof(MIMETYPE) + mimeTypeStringsSize) * numTypes;
char* data = (char*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, totalSize);
MIMETYPE* mimeTypes = (MIMETYPE*)data;
char* stringData = data + (sizeof(MIMETYPE) * numTypes);
for(unsigned int i = 0; i < numTypes; i++)
{
//get data from file
getline(fin, line);
stringstream parser.str(line);
parser >> ext >> type;
// set pointers to proper locations
mimeTypes[i].extension = stringData + (mimeTypeStringsSize * i);
mimeTypes[i].type = stringData + (mimeTypeStringsSize * i) + maxExtension+1;
//copy the data into the elements
strcpy(mimeTypes[i].extension, ext.c_str());
strcpy(mimeTypes[i].type, type.c_str());
}
(Note: I've based my byte layout explanations on typical behavior of 32-bit code. 64-bit code would have more space used for the pointers, but the principle is the same. Furthermore, the actual code I've written here should work regardless of 32/64-bit differences.)
What you need to do is get a garbage collector and manage the heap. A simple collector using RAII for object destruction is not that difficult to write. That way, you can simply allocate off the collector and know that it's going to be contiguous. However, you should really, REALLY profile before determining that this is a serious problem for you. When that happens, you can typedef many std types like string and stringstream to use your custom allocator, meaning that you can go back to just std::string instead of the C-style string horrors you have there.
You really have to know the length of extension and type in order to allocate MIMETYPEs contiguously (if "contiguously" means that extension and type are actually allocated within the object). Since you say that the length of extension and type are not known at compile time, you cannot do this in an array or a vector (the overall length of a vector can be set and changed at runtime, but the size of the individual elements must be known at compile time, and you can't know that size without knowing the length of extension and type).
I would personally recommend using a vector of MIMETYPEs, and making the extension and type fields both strings. You're requirements sound suspiciously like premature optimization guided by a gut feeling that dereferencing pointers is slow, especially if the pointers cause cache misses. I wouldn't worry about that until you have actual data that reading these fields is an actual bottleneck.
However, I can think of a possible "solution": you can allocate the extension and type strings inside the MIMETYPE object when they are shorter than a particular threshold and allocate them dynamically otherwise:
#include <algorithm>
#include <cstring>
#include <new>
template<size_t Threshold> class Kinda_contig_string {
char contiguous_buffer[Threshold];
char* value;
public:
Kinda_contig_string() : value(NULL) { }
Kinda_contig_string(const char* s)
{
size_t length = std::strlen(s);
if (s < Threshold) {
value = contiguous_buffer;
}
else {
value = new char[length];
}
std::strcpy(value, s);
}
void set(const char* s)
{
size_t length = std::strlen(s);
if (length < Threshold && value == contiguous_buffer) {
// simple case, both old and new string fit in contiguous_buffer
// and value points to contiguous_buffer
std::strcpy(contiguous_buffer, s);
return;
}
if (length >= Threshold && value == contiguous_buffer) {
// old string fit in contiguous_buffer, new string does not
value = new char[length];
std::strcpy(value, s);
return;
}
if (length < Threshold && value != contiguous_buffer) {
// old string did not fit in contiguous_buffer, but new string does
std::strcpy(contiguous_buffer, s);
delete[] value;
value = contiguous_buffer;
return;
}
// old and new strings both too long to fit in extension_buffer
// provide strong exception guarantee
char* temp_buffer = new char[length];
std::strcpy(temp_buffer, s);
std::swap(temp_buffer, value);
delete[] temp_buffer;
return;
}
const char* get() const
{
return value;
}
}
class MIMETYPE {
Kinda_contig_string<16> extension;
Kinda_contig_string<64> type;
public:
const char* get_extension() const
{
return extension.get();
}
const char* get_type() const
{
return type.get();
}
void set_extension(const char* e)
{
extension.set(e);
}
// t must be NULL terminated
void set_type(const char* t)
{
type.set(t);
}
MIMETYPE() : extension(), type() { }
MIMETYPE(const char* e, const char* t) : extension(e), type(t) { }
};
I really can't endorse this without feeling guilty.
Add one byte in between strings... extension and type are not \0-terminated the way do it.
here you allocate allowing for an extra \0 - OK
unsigned int size = (maxExtension+1 + maxType+1) * numTypes;
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size);
here you don't leave any room for extension's ending \0 (if string len == maxExtension)
//point the pointers at a spot in the memory that I allocated
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension);
instead i think it should be
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension + 1);