This questions has good answers on how to write an std::vector into a file: Reading and writing a std::vector into a file correctly
In my case, I have a vector of arrays:
vector<array<double, 3> > vec;
I would like to write into a file in order to get a file having the following format, where the values are doubles and the first number is the position in the vector and the second is the position in the array:
vec0_0 vec0_1 vec0_2 vec1_0 vec1_1 vec1_2 vec2_0 ...
Can I just use...
std::copy(vec.begin(), vec.end(), std::ostreambuf_iterator<char>(FILE));
...or...
size_t sz = vec.size();
FILE.write(reinterpret_cast<const char*>(&vec[0]), sz * sizeof(vec[0]));
...as proposed in the mentioned question for a scalar type, or do I need to do it differently because the type in the vector is an array?
From what I understand, std::array has contiguous storage. However, I don't think that guarantees there is no padding. If that were just a double[3], it would work out of the box, but I think you'd have to test very carefully and worry about portability with a std::array inside the container.
In fact, looking around there is already an example out there of a system that pads.
std::array alignment
sizeof(int) = 4;
sizeof( std::tr1::array< int,3 > ) = 16;
sizeof( std::tr1::array< int,4 > ) = 16;
sizeof( std::tr1::array< int,5 > ) = 32;
Presumably this padding is implementation defined, or maybe you can find it in the standard somewhere. In any case, I'd just iterate the thing or use a non-stl array.
I'd guess the concept is similar to a struct where there is often padding introduced to optimize memory access, however the compiler is optimizing that padding, and it can be turned off on most compilers with #pragma pack statements. Not true of stl containers to my knowledge.
I need a structure as follow:
The structure must hold fixed size std::strings so that the number of its elements is finit (100 - 10000000).
I would like to be able to access each element randomly as follow:
std::string Temp = MyStrcuture[i];
or
MyStrcuture[i] = std::string Temp;
I have to use the fastest structure with no (possibly) memory leak.
Which one is better for me?
std::string* MyStrcuture = new std::string[Nu_of_Elements];
std::queue< std:string> MyStrcuture(Nu_of_Elements);
std::vector< std:string> MyStrcuture(Nu_of_Elements);
boost::circular_buffer< std::string> MyStrcuture(Nu_of_Elements);
Your suggestion?
std::vector< std:string> MyStrcuture(Nu_of_Elements);
Vector is the best fit for your requirements. It supports index-based element access as the elements are stored in continuous memory addresses, and has flexibility with size.
std:string* MyStrcuture = new std::string[Nu_of_Elements]; No
C++ STL vector vs array in the real world
std::queue< std:string> MyStrcuture(Nu_of_Elements); No
How do I get the nth item in a queue in java?
Index-based element access is not supported.
std::vector< std:string> MyStrcuture(Nu_of_Elements); Yes
Clean-up : The vector's destructor automatically invokes the destructor of each element in the vector.
Boost::circular_buffer< std::string> MyStrcuture(Nu_of_Elements); No
Same reason as second one. Know more
Well, since your string have fixed size, if you don't have dedicated requirement when processing string and have enough free memory for contiguous allocation. You can use std::array< char, 400 > or std::unique_ptr< char* > instead of std::string.
You have to manage memory in C way. consider smart pointer
std::queue doesn't have random access, Access c++ queue elements like an array
std::vector is suitable if the number of string will be changed. However, the clear() function just call the destructor of elements, not free vector allocated memory (you can check the capacity after clear).
After reading boost documentation. The random access circular buffer is suitable if your number of string have an upper limit (that you said 10 millions). But its a waste of memory if actually you have so few strings. So I suggest to use with smart pointer.
If your number of string are fixed and unchanged from the beginning. You can have a look at C++11 array container
If number of elements and length is fixed and memory is critical, you may consider using plain char array, which provides minimal memory overhead and fast accessibility. Your code will look like this:
char* MyStructure = new char[n * 401];
memset(MyStructure, 0, n * 401);
std::string Temp = MyStructure[i * 401]; // Get value
strcpy(MyStructure[i * 401], Temp.c_str()); // Put value
401 here is for 400 bytes of your string and 1 trailing zero.
Is the following code ok?:
std::vector<char> var;
size_t requiredSize;
getenv_s(&requiredSize, NULL, 0, "Something");
if (requiredSize == 0)
{
return ENV_NOT_EXIST;
}
if(var.size() < requiredSize)
var.resize(requiredSize);
// Get the value of the environment variable.
getenv_s(&requiredSize, &var[0], requiredSize, "Something");
std::string str(var.begin(),var.end());
If this code is OK, can someone please explain me how the begin() and the end() values of the var vector are updated? it looks like this code changes directly the internal array of the vector, not over the std::vector api - so how these values are updated to the actual size?
std::vector guarantees data to be stored contiguously, so writing to data, as long as you do not overrun the end is perfectly fine:
From the C++11 standard section 23.3.6.1.1:
The elements of a vector are stored contiguously, meaning that if v is
a vector where T is some type other than bool, then it
obeys the identity &v[n] == &v[0] + n for all 0 <= n < v.size().
However, note that resizing the vector might move the data and invalidate iterators.
Unfortunately, the standard does not require std::vector<T>::iterator to be a raw pointer type (although it usually is). So, you cannot portably use std::vector<T>::begin() to access the first element. There is std::vector<T>::data(), which returns a pointer to the first element and which can be used for code that expects raw c-arrays.
I suggest to rewrite your call like this:
getenv_s(&requiredSize, var.data(), var.size(), "Something");
if (requiredSize < var.size())
var.resize(requiredSize);
I've read file contents into a char array, and then read some data of it into a vector.
How can i copy a range of the char array into the vector? both vector and char array is the same type (unsigned char).
Current code goes something like this:
int p = 0;
for(...){
short len = (arr[p+1] << 8) | arr[p+0];
p+=2;
...
for(...len...){
vec.push_back(arr[p]);
p++;
}
}
I would like to improve this by dropping the loop with push_back, How?
Appending something to a vector can be done using the insert() member function:
vec.insert(vec.end(), arr, arr+len);
Of course, there's also an assign(), which is probably closer to what you want to do:
vec.assign(arr, arr+len);
However, reading your question I wondered why you would first read into a C array just to copy its content into a vector, when you could read into a vector right away. A std::vector<> is required to keep its data in one contiguous block of memory, and you can access this block by taking the address of its first element. Just make sure you have enough room in the vector:
std::size_t my_read(char* buffer, std::size_t buffer_size);
vec.resize( appropriate_length );
vec.resize( my_read_func(&vec[0], vec.size()) );
Instead of &vec[0] you could also get the address of the first element by &*vec.begin(). However, note that with either method you absolutely must make sure there's at least one element in the vector. None of the two methods are required to check for it (although your implementation might do so for debug builds), and both will invoke the dreaded Undefined Behavior when you fail on this.
I have an array of values that is passed to my function from a different part of the program that I need to store for later processing. Since I don't know how many times my function will be called before it is time to process the data, I need a dynamic storage structure, so I chose a std::vector. I don't want to have to do the standard loop to push_back all the values individually, it would be nice if I could just copy it all using something similar to memcpy.
There have been many answers here and just about all of them will get the job done.
However there is some misleading advice!
Here are the options:
vector<int> dataVec;
int dataArray[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
unsigned dataArraySize = sizeof(dataArray) / sizeof(int);
// Method 1: Copy the array to the vector using back_inserter.
{
copy(&dataArray[0], &dataArray[dataArraySize], back_inserter(dataVec));
}
// Method 2: Same as 1 but pre-extend the vector by the size of the array using reserve
{
dataVec.reserve(dataVec.size() + dataArraySize);
copy(&dataArray[0], &dataArray[dataArraySize], back_inserter(dataVec));
}
// Method 3: Memcpy
{
dataVec.resize(dataVec.size() + dataArraySize);
memcpy(&dataVec[dataVec.size() - dataArraySize], &dataArray[0], dataArraySize * sizeof(int));
}
// Method 4: vector::insert
{
dataVec.insert(dataVec.end(), &dataArray[0], &dataArray[dataArraySize]);
}
// Method 5: vector + vector
{
vector<int> dataVec2(&dataArray[0], &dataArray[dataArraySize]);
dataVec.insert(dataVec.end(), dataVec2.begin(), dataVec2.end());
}
To cut a long story short Method 4, using vector::insert, is the best for bsruth's scenario.
Here are some gory details:
Method 1 is probably the easiest to understand. Just copy each element from the array and push it into the back of the vector. Alas, it's slow. Because there's a loop (implied with the copy function), each element must be treated individually; no performance improvements can be made based on the fact that we know the array and vectors are contiguous blocks.
Method 2 is a suggested performance improvement to Method 1; just pre-reserve the size of the array before adding it. For large arrays this might help. However the best advice here is never to use reserve unless profiling suggests you may be able to get an improvement (or you need to ensure your iterators are not going to be invalidated). Bjarne agrees. Incidentally, I found that this method performed the slowest most of the time though I'm struggling to comprehensively explain why it was regularly significantly slower than method 1...
Method 3 is the old school solution - throw some C at the problem! Works fine and fast for POD types. In this case resize is required to be called since memcpy works outside the bounds of vector and there is no way to tell a vector that its size has changed. Apart from being an ugly solution (byte copying!) remember that this can only be used for POD types. I would never use this solution.
Method 4 is the best way to go. It's meaning is clear, it's (usually) the fastest and it works for any objects. There is no downside to using this method for this application.
Method 5 is a tweak on Method 4 - copy the array into a vector and then append it. Good option - generally fast-ish and clear.
Finally, you are aware that you can use vectors in place of arrays, right? Even when a function expects c-style arrays you can use vectors:
vector<char> v(50); // Ensure there's enough space
strcpy(&v[0], "prefer vectors to c arrays");
If you can construct the vector after you've gotten the array and array size, you can just say:
std::vector<ValueType> vec(a, a + n);
...assuming a is your array and n is the number of elements it contains. Otherwise, std::copy() w/resize() will do the trick.
I'd stay away from memcpy() unless you can be sure that the values are plain-old data (POD) types.
Also, worth noting that none of these really avoids the for loop--it's just a question of whether you have to see it in your code or not. O(n) runtime performance is unavoidable for copying the values.
Finally, note that C-style arrays are perfectly valid containers for most STL algorithms--the raw pointer is equivalent to begin(), and (ptr + n) is equivalent to end().
If all you are doing is replacing the existing data, then you can do this
std::vector<int> data; // evil global :)
void CopyData(int *newData, size_t count)
{
data.assign(newData, newData + count);
}
std::copy is what you're looking for.
Since I can only edit my own answer, I'm going to make a composite answer from the other answers to my question. Thanks to all of you who answered.
Using std::copy, this still iterates in the background, but you don't have to type out the code.
int foo(int* data, int size)
{
static std::vector<int> my_data; //normally a class variable
std::copy(data, data + size, std::back_inserter(my_data));
return 0;
}
Using regular memcpy. This is probably best used for basic data types (i.e. int) but not for more complex arrays of structs or classes.
vector<int> x(size);
memcpy(&x[0], source, size*sizeof(int));
int dataArray[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };//source
unsigned dataArraySize = sizeof(dataArray) / sizeof(int);
std::vector<int> myvector (dataArraySize );//target
std::copy ( myints, myints+dataArraySize , myvector.begin() );
//myvector now has 1,2,3,...10 :-)
Yet another answer, since the person said "I don't know how many times my function will be called", you could use the vector insert method like so to append arrays of values to the end of the vector:
vector<int> x;
void AddValues(int* values, size_t size)
{
x.insert(x.end(), values, values+size);
}
I like this way because the implementation of the vector should be able to optimize for the best way to insert the values based on the iterator type and the type itself. You are somewhat replying on the implementation of stl.
If you need to guarantee the fastest speed and you know your type is a POD type then I would recommend the resize method in Thomas's answer:
vector<int> x;
void AddValues(int* values, size_t size)
{
size_t old_size(x.size());
x.resize(old_size + size, 0);
memcpy(&x[old_size], values, size * sizeof(int));
}
avoid the memcpy, I say. No reason to mess with pointer operations unless you really have to. Also, it will only work for POD types (like int) but would fail if you're dealing with types that require construction.
In addition to the methods presented above, you need to make sure you use either std::Vector.reserve(), std::Vector.resize(), or construct the vector to size, to make sure your vector has enough elements in it to hold your data. if not, you will corrupt memory. This is true of either std::copy() or memcpy().
This is the reason to use vector.push_back(), you can't write past the end of the vector.
Assuming you know how big the item in the vector are:
std::vector<int> myArray;
myArray.resize (item_count, 0);
memcpy (&myArray.front(), source, item_count * sizeof(int));
http://www.cppreference.com/wiki/stl/vector/start