My task is to measure time of communication betweeen two processes.
I want to send 4,8,...,1000,...., 10000 bytes of data and measure time it takes to send and receive back the message.
So i figured out that i will send an array of shorts.
When i send array initialised like that:
mpi::communicator world;
short message[100000];
....
world.send(1,0, message);
time seems to be ok, and I can see a time difference between message[100000] and [1000]
But I want to allocate array dynamically like that:
short *message = new short[100000];
...
world.send(1,0, *message);
It seems like the second send is always sending the same amount of data no matter what size the array will be.
So my question is, how to send a dynamically allocated array?
In the second case message is of type short * and *message dereferences to a scalar short, i.e. to the first element of the array only. Use
world.send(1, 0, message, n);
instead and vary the value of n. It should also (probably) work if you cast the pointer to a pointer to an array and then dereference it:
world.send(1, 0, *reinterpret_cast<int(*)[100]>(message));
The int(*)[100] type is a pointer to an integer array with 100 elements.
Related
How to get the size of datatype of an array? For Eg. int32_t arr[10][10] datatype is int32_t and it's size is 4 bytes, similarly for char [10][100] datatype is char and it's size is 1 byte. The reason I'm asking this question is I need to know whether data has to be swapped or not before sending via network, since datatypes of size greater than 1 need to swapped and datatypes of size equal to 1 shouldn't be swapped(in fact swapping doesn't make sense).
int32_t int_arr[10][10][10]; // execution should go to if part
if (sizeof(decltype(int_arr)) > 1) // this doesn't work for array type
// swap each element and then copy to send buff
else
// don't swap, simply copy to send buff
char char_arr[10][10]; // execution should go to else part
if (sizeof(decltype(char_arr)) > 1) // this doesn't work for array type
// swap each element and then copy to send buff
else
// don't swap, simply copy to send buff
You may be looking for
sizeof(std::remove_all_extents_t<decltype(int_arr)>)
Works for non-arrays, too; when given a non-array type T, std::remove_all_extents_t<T> is simply T.
I have a vector<vector <string>> a; How could I pass it to the enclave? How I declare edl function.
A sample function declaration for the app, edl and enclave is much appreciated.
I am aware of this: C++ Arguments to SGX Enclave Edge Functions.
A sample to pass even a vector<string> is ok for me.
update1:
I came up with this:
App.cpp
const char *convert(const std::string & s)
{
return s.c_str();
}
vector<string> members_data;
member_data.push_back("apple");
member_data.push_back("orange"); //just for sample
std::vector<const char*> vc;
std::transform(members_data.begin(), members_data.end(), std::back_inserter(vc), convert);
edl:
trusted {
public void ecall_receive_vector([in, size=len] const char **arr, size_t len);
};
enclave
void ecall_receive_vector(const char *arr[], size_t len)
{
vector<string> v(arr, arr+len);
printf("%s\n", v[2].c_str());
}
But enclave does not receive any data, the program compiles perfectly with no error. Could anyone help? The printf is the sample ocall.
In the EDL use count instead of size.
trusted {
public void ecall_receive_vector([in, count=len] const char **arr, size_t len);
};
You are passing a double pointer, it is, a pointer to pointer to char (char **).
While marshaling/unmarshaling pointers, the EDL Processor processes (copies and validates input and output) only the first level of indirection, it's up to the developer to handle the additional levels of indirection. Hence, for an array of pointers it will only copy the first array of pointers, not the pointed values, copying them is the developer's responsibility.
If not specified count and size default to 1 and sizeof(<pointed-type>) respectively. In your case size = sizeof(<pointer>) which in most platforms is 4.
In your case, you provided only size. As you don't provide the caller code I assume you're passing the length of the string, and as count was not specified it defaults to 1. Then the total number of bytes, based on Total number of bytes = count * size will be 1 * len which is wrong.
Using only count will let size default to sizeof(<pointed-type>), then Total number of bytes = count * size will be count * sizeof(<pointed-type>), which is right because you're passing an array of pointers.
To close, once inside the Enclave you need to copy the pointers' data because those pointers reside out of the enclave, that may be done automatically by assigning them to a std::string.
From Intel SGX SDK Documentation:
Pointer Handling (the last paragraph)
You may use the direction attribute to trade protection for performance. Otherwise, you must use the user_check attribute described below and validate the data obtained from untrusted memory via pointers before using it, since the memory a pointer points to could change unexpectedly because it is stored in untrusted memory. However, the direction attribute does not help with structures that contain pointers. In this scenario, developers have to validate and copy the buffer contents, recursively if needed, themselves.
And,
Buffer Size Calculation
The generalized formula for calculating the buffer size using these attributes:
Total number of bytes = count * size
The above formula holds when both count and size/sizefunc are specified.
size can be specified by either size or sizefunc attribute.
If count is not specified for the pointer parameter, then it is assumed to be equal to 1, i.e., count=1. Then total number of bytes equals to size/sizefunc.
If size is not specified, then the buffer size is calculated using the above formula where size is sizeof (element pointed by the pointer).
Over time I am receiving parts(RdKafka::Message) of a larger message and storing them in
std::vector<std::unique_ptr<RdKafka::Message>> messageParts;
The RdKafka::Message has a raw pointer payload that I want to move into a single shared_ptr. I can do this because I have the size and index into the original larger message contained in the RdKafka::Message.
Once I have received all parts of the larger message I want to reassemble all of the messages into a single shared_ptr.
My first strategy has been to allocate the total amount of memory space in a C style pointer and then std::move each individual unique_ptr part of the data into the correct section of the C style memory. Something like this:
uint8_t* payload = new uint8_t[totalBytes];
for(auto& m : messageParts){
int32_t partition = m->partition();
size_t len = m->len();
std::memcpy(&payload[partition], (uint8_t*)(m->payload()), len);
}
std::shared_ptr<uint8_t[]>(payload);
This obviously doesn't compile but I am looking for something like this so that I can return a single pointer to all the original memory.
I have an array of character pointers which I want to send to device. Can somebody tell me how?
Here is what I have tried so far:
char **a;
char **b;
*a[0]="Foo1";
*a[1]=="Foo2";
cudaMalloc(void**)?,sizeof(?);
cudamemcpy(b,a,sizeof(?),cudaMemcpyHostToDevice);
How do I pass in the parameters to the above two functions?
And finally how should the kernel be called? (Do I just pass b or *b or something?)
If you send the character pointers to the device, you will have an array of CPU memory addresses on the device, which is probably not what you want.
If you want to send the whole data structure there, allocate sizeof(char) * string_length bytes for each string, and then store the resulting device pointers in a CPU array of char*s. Then, once it's complete, send the array of device pointers to the device, allocating sizeof(char*) * number_of_strings bytes for it.
When you call the kernel, give it the device-side array of device pointers.
to assign, use array[0] = "string literal"
No need for stars.
To get length, use strlen(). siezeof is irrelevant.
Never copy into this string matrix, or pass it as out parameter.
You have to allocate memory for that.
I need to be able to set the size of an array based on the number of bytes in a file.
For example, I want to do this:
// Obtain the file size.
fseek (fp, 0, SEEK_END);
size_t file_size = ftell(fp);
rewind(fp);
// Create the buffer to hold the file contents.
char buff[file_size];
However, I get a compile time error saying that the size of the buffer has to be a constant.
How can I accomplish this?
Use a vector.
std::vector<char> buff(file_size);
The entire vector is filled with '\0' first, automatically. But the performance "lost" might not be noticable. It's certainly safer and more comfortable. Then access it like a usual array. You may even pass the pointer to the data to legacy C functions
legacy(&buff[0]); // valid!
You should use a std::vector and not an array.
Real arrays require you to specify their size so that the compiler can create some space for them -- this is why the compiler complains when you don't supply a constant integer. Dynamic arrays are represented by a pointer to the base of the array -- and you have to retrieve the memory for the dynamic array yourself. You may then use the pointer with subscript notation. e.g.,
int * x;
x = (int *) malloc( sizeof(int) *
getAmountOfArrayElements() /* non-const result*/
);
x[5] = 10;
This leads to two types of problems:
Buffer over/under flows : you might subscript-index past either end of the array.
You might forget to release the memory.
Vector provides a nice little interface to hide these problems from you -- if used correctly.
Replace
char buff[file_size];
with
char *buff = new char[file_size];
and once the use of the buff is done..you can free the memory using:
delete[] buff;
There are two points in your question I'd like to cover.
The actual question, how do you create the array. Johannes answered this. You use a std::vector and create it with a size allocation.
Your error message. When you declare an array of some type, you must declare it with a constant size. So for example
const int FileSize = 1000;
// stuff
char buffer[FileSize];
is perfectly legitimate.
On the other hand, what you did, attempting to declare an array with variable size, and then not allocating with new, generates an error.
Problem is that buff needs be created on the heap (instead of stack). Compiler want s to know the exact size to create on the stack.
char* buff = new char[file_size];