Large vector "Segmentation fault" error - c++

I have gathered a large amount of extremely useful information from other peoples' questions and answers on SO, and have searched duly for an answer to this one as well. Unfortunately I have not found a solution to this problem.
The following function to generate a list of primes:
void genPrimes (std::vector<int>* primesPtr, int upperBound = 10)
{
std::ofstream log;
log.open("log.txt");
std::vector<int>& primesRef = *primesPtr;
// Populate primes with non-neg reals
for (int i = 2; i <= upperBound; i++)
primesRef.push_back(i);
log << "Generated reals successfully." << std::endl;
log << primesRef.size() << std::endl;
// Eratosthenes sieve to remove non-primes
for (int i = 0; i < primesRef.size(); i++) {
if (primesRef[i] == 0) continue;
int jumpStart = primesRef[i];
for (int jump = jumpStart; jump < primesRef.size(); jump += jumpStart) {
if (primesRef[i+jump] == 0) continue;
primesRef[i+jump] = 0;
}
}
log << "Executed Eratosthenes Sieve successfully.\n";
for (int i = 0; i < primesRef.size(); i++) {
if (primesRef[i] == 0) {
primesRef.erase(primesRef.begin() + i);
i--;
}
}
log << "Cleaned list.\n";
log.close();
}
is called by:
const int SIZE = 500;
std::vector<int>* primes = new std::vector<int>[SIZE];
genPrimes(primes, SIZE);
This code works well. However, when I change the value of SIZE to a larger number (say, 500000), the compiler returns a "segmentation error." I'm not familiar enough with vectors to understand the problem. Any help is much appreciated.

You are accessing primesRef[i + jump] where i could be primesRef.size() - 1 and jump could be primesRef.size() - 1, leading to an out of bounds access.
It is happening with a 500 limit, it is just that you happen to not have any bad side effects from the out of bound access at the moment.
Also note that using a vector here is a bad choice as every erase will have to move all of the following entries in memory.

Are you sure you wanted to do
new std::vector<int> [500];
and not
new std::vector<int> (500);
In the latter case, you are specifying the size of the vector, whose location is available to you via the variable named 'primes'.
In the former, you are requesting space for 500 vectors, each sized to the default that the STL library wants.
That would be something like (on my system : 24*500 bytes). In the latter case, 500 length vector(only one vector) is what you are asking for.
EDIT: look at the usage - he needs just one vector.
std::vector& primesRef = *primesPtr;

The problem lies here:
// Populate primes with non-neg reals
for (int i = 2; i <= upperBound; i++)
primesRef.push_back(i);
You only have N-2 elements in your vector pushed back, but then try to access an element at N-1 (i+jump). The fact that it did not fail on 500 is just dumb luck that the memory being overwritten was not catastrophic.

This code works well. However, when I change the value of SIZE to a larger number (say, 500000), ...
That may blow your stack, and be to big allocated with it. You need dynamic memory allocation for all of the std::vector<int> instances you believe to need.
To achieve that, simply use a nested std::vetcor like this.
std::vector<std::vector<int>> primes(SIZE);
instead.
But to get straight on, I seriously doubt you need number of SIZE vector instances to store all of the prime numbers found, but just a single one initialized like this:
std::vector<int> primes(SIZE);

Related

How to choose a random number excluding those which were previously chosen? [duplicate]

I'd like to make a number generator that does not repeat the number it has given out
already (C++).
All I know is:
int randomgenerator(){
int random;
srand(time(0));
random = rand()%11;
return(random);
} // Added this on edition
That function gives me redundant numbers.
I'm trying to create a questionnaire program that gives out 10 questions in a random order and I don't want any of the questions to reappear.
Does anyone know the syntax?
What I would do:
Generate a vector of length N and fill it with values 1,2,...N.
Use std::random_shuffle.
If you have say 30 elements and only want 10, use the first 10 out the vector.
EDIT: I have no idea how the questions are being stored, so.. :)
I am assuming the questions are being stored in a vector or somesuch with random access. Now I have generated 10 random numbers which don't repeat: 7, 4, 12, 17, 1, 13, 9, 2, 3, 10.
I would use those as indices for the vector of questions:
std::vector<std::string> questions;
//fill with questions
for(int i = 0; i < number_of_questions; i++)
{
send_question_and_get_answer(questions[i]);
}
You are trying to solve the problem "the wrong way".
Try this instead (supposing you have a vector<int> with question ids, but the same idea will work with whatever you have):
Get a random R from 0 to N-1 where N is the number of questions in the container
Add question R to another collection of "selected" questions
If the "selected questions" collection has enough items, you 're done
Remove question R from your original container (now N has decreased by 1)
Go to 1
Sounds like you essentially want to shuffle a deck of cards (in this case, the "cards" being the questions, or question numbers).
In C++, I would do:
#include <vector>
#include <algorithms>
std::vector<int> question_numbers;
for (unsigned int i = 0; i < 10; ++i)
question_numbers.push_back(i+1);
std::random_shuffle(question_numbers.begin(), question_numbers.end());
// now dole out the questions based on the shuffled numbers
You do not have to hand out all of the questions, any more than you have to deal out a whole deck of cards every time you play a game. You can, of course, but there's no such requirement.
Create a vector of 10 elements (numbers 1-10), then shuffle it, with std::random_shuffle. Then just iterate through it.
Should look more like this: (Note: does not solve your original problem).
int randomgenerator(){
int random;
// I know this looks re-dunand compared to %11
// But the bottom bits of rand() are less random than the top
// bits do you get a better distribution like this.
random = rand() / (RAND_MAX / 11);
return random;
}
int main()
{
// srand() goes here.
srand(time(0));
while(true)
{
std::cout << randomgenerator() << "\n";
}
}
A better way to solve the original problem is to pre-generate the numbers so you know that each number will appear only once. Then shuffle the order randomly.
int main()
{
int data[] = { 0,1,2,3,4,5,6,7,8,9,10,11};
int size = sizeof(data)/sizeof(data[0]);
std::random_shuffle(data, data + size);
for(int loop = 0; loop < size; ++loop)
{
std::cout << data[loop] << "\n";
}
}
Why not use some STL to perform the checks for you? The idea:
Create an (initially empty) set of 10 integers that will be the indices of the random questions (they will be distinct as a set forbids duplicate items). Keep pushing random numbers in [0, num_of_questions-1] in there until it grows to a size of 10 (duplicates will get rejected automatically). When you have that set ready, iterate over it and output the questions of the corresponding indexes:
std::vector<std::string> questions = /* I am assuming questions are stored in here */
std::set<int> random_indexes;
/* loop here until you get 10 distinct integers */
while (random_indexes.size() < 10) random_indexes.insert(rand() % questions.size());
for (auto index: random_indexes){
std::cout << questions[index] <<std::endl;
}
I may be missing something, but it seems to me the answers that use shuffling of either questions or indexes perform more computations or use an unnecessary memory overhead.
//non repeating random number generator
for (int derepeater = 0; derepeater < arraySize; derepeater++)
{
for (int j = 0; j < arraySize; j++)
{
for (int i = arraySize; i > 0; i--)
{
if (Compare[j] == Compare[i] && j != i)
{
Compare[j] = rand() % upperlimit + 1;
}
}
}
}

luaT_pushudata returns proper Tensor type and dimension, but garbage data

I have a short clip of C++ code that should theoretically work to create and return a torch.IntTensor object, but when I call it from Torch I get garbage data.
Here is my code (note this snippet leaves out the function registering, but suffice it to say that it registers fine--I can provide it if necessary):
static int ltest(lua_State* L)
{
std::vector<int> matches;
for (int i = 0; i < 10; i++)
{
matches.push_back(i);
}
performMatching(dist, matches, ratio_threshold);
THIntStorage* storage = THIntStorage_newWithData(&matches[0], matches.size());
THIntTensor* tensorMatches = THIntTensor_newWithStorage1d(storage, 0, matches.size(), 1);
// Push result to Lua stack
luaT_pushudata(L, (void*)tensorMatches, "torch.IntTensor");
return 1;
}
When I call this from Lua, I should get a [torch.IntTensor of size 10] and I do. However, the data appears to be either memory addresses or junk:
29677072
0
16712197
3
0
0
29677328
0
4387616
0
[torch.IntTensor of size 10]
It should have been the numbers [0,9].
Where am I going wrong?
For the record, when I test it in C++
for (int i = 0; i < storage->size; i++)
std::cout << *(storage->data+i) << std::endl;
prints the proper values.
As does
for (int i = 0; i < tensorMatches->storage->size; i++)
std::cout << *(tensorMatches->storage->data+i) << std::endl;
so it seems clear to me that the problem lies in the exchange between C++ and Lua.
So I got an answer elsewhere--the Google group for Torch7--but I'll copy and paste it here for anyone who may need it.
From user #alban desmaison:
Your problem is actually memory management.
When your C++ function return, you vector<int> is free, and so is its content.
From that point onward, the tensor is pointing to free memory and when you access it, you access freed memory.
You will have to either:
Allocate memory on the heap with malloc (as an array of ints) and use THIntStorage_newWithData as you currently do (the pointer that you give to newWithData will be freeed when it is not used anymore by Torch).
Use a vector<int> the way you currently do but create a new Tensor with a given size with THIntTensor_newWithSize1d(matches.size()) and then copy the content of the vector into the tensor.
For the record, I couldn't get it to work with malloc but the copying memory approach worked just fine.

Writing two versions of a function, one for "clarity" and one for "speed"

My professor assigned homework to write a function that takes in an array of integers and sorts all zeros to the end of the array while maintaining the current order of non-zero ints. The constraints are:
Cannot use the STL or other templated containers.
Must have two solutions: one that emphasizes speed and another that emphasizes clarity.
I wrote up this function attempting for speed:
#include <iostream>
#include <cstdio>
#include <cstdlib>
using namespace std;
void sortArray(int array[], int size)
{
int i = 0;
int j = 1;
int n = 0;
for (i = j; i < size;)
{
if (array[i] == 0)
{
n++;
i++;
}
else if (array[i] != 0 && j != i)
{
array[j++] = array[i++];
}
else
{
i++;
n++;
}
}
while (j < size)
{
array[j++] = 0;
}
}
int main()
{
//Example 1
int array[]{20, 0, 0, 3, 14, 0, 5, 11, 0, 0};
int size = sizeof(array) / sizeof(array[0]);
sortArray(array, size);
cout << "Result :\n";
for (int i = 0; i < size; i++)
{
cout << array[i] << " ";
}
cout << endl << "Press any key to exit...";
cin.get();
return 0;
}
It outputs correctly, but;
I don't know what the speed of it actually is, can anyone help me figure out how to calculate that?
I have no idea how to go about writing a function for "clarity"; any ideas?
I my experience, unless you have very complicated algorithm, speed and clarity come together:
void sortArray(int array[], int size)
{
int item;
int dst = 0;
int src = 0;
// collect all non-zero elements
while (src < size) {
if (item = array[src++]) {
array[dst++] = item;
}
}
// fill the rest with zeroes
while (dst < size) {
array[dst++] = 0;
}
}
Speed comes from a good algorithm. Clarity comes from formatting, naming variables and commenting.
Speed as in complexity?
Since you are, and need, to look at all the elements in the array — and as such have a single loop going through the indexes in the range [0, N)—where N denotes the size of the input—your solution is O(N).
Further reading:
Plain English explanation of big O
Determining big O Notation
Regarding clearity
In my honest opinion there shouldn't need to be two alternatives when implementing such functionality as you are presenting. If you rename your variables to more suitable (descriptive) names your current solution should be clear enough to count as both performant and clear.
Your current approach can be written in plain english in a very clear fashion:
pseudo-explanation
set write_index to 0
set number_of_zeroes to 0
For each element in array
If element is 0
increase number_of_zeros by one
otherwise
write element value to position denoted by write_index
increase write_index by one
write number_of_zeroes 0s at the end of array
Having stated the explanation above we can quickly see that sortArray is not a descriptive name for your function, a more suitable name would probably be partition_zeroes or similar.
Adding comments could improve readability, but you current focus should lie in renaming your variables to better express the intent of the code.
(I feel your question is almost off-topic; I am answering it from a Linux perspective; I recommend using Linux to learn C++ programming; you'll adapt my advices to your operating system if you are using something else....)
speed
Regarding speed, you should have two complementary approaches.
The first (somehow "theoretical") is to analyze (i.e. think on) your algorithm and give (with some proof) its asymptotic time complexity.
The second approach (only "practical", and often pragmatical) is to benchmark and profile your program. Don't forget to compile with optimizations enabled (e.g. using g++ -Wall -O2 with GCC). Have a benchmark which runs for more than half of a second (so processes a large amount of data, e.g. several million numbers) and repeat it several times (e.g. using time(1) command on Linux). You could also measure some time inside your program using e.g. <chrono> in C++11, or just clock(3) (if you read a large array from some file, or build a large array of pseudo-random numbers with <random> or with random(3) you certainly want to measure separately the time to read or fill the array with the time to move zeros out of it). See also time(7).
(You need to process a large amount of data - more than a million items, perhaps many millions of them - because computer are very fast; a typical "elementary" operation -a machine instruction- takes less than a nanosecond, and you have lot of uncertainty on a single run, see this)
clarity
Regarding clarity, it is a bit subjective, but you might try to make your code readable and concise. Adding a few good comments could also help.
Be careful about naming: sorting is not exactly what your program is doing (it is more moving zeros than sorting the array)...
I think this is the best - Of course you may wish to use doxygen or some other
// Shift the non-zeros to the front and put zero in the rest of the array
void moveNonZerosTofront(int *list, unsigned int length)
{
unsigned int from = 0, to = 0;
// This will move the non-zeros
for (; from < length; ++from) {
if (list[from] != 0) {
list[to] = list[from];
to++;
}
}
// So the rest of the array needs to be assigned zero (as we found those on the way)
for (; to < length; +=to) {
list[to] = 0;
}
}

Vector + for + if

OK, so the goal of this was to write some code for the Fibonacci numbers itself then take those numbers figure out which ones were even then add those specific numbers together. Everything works except I tried and tried to figure out a way to add the numbers up, but I always get errors and am stumped as of how to add them together. I looked elsewhere but they were all asking for all the elements in the vector. Not specific ones drawn out of an if statement.
P.S. I know system("pause") is bad but i tried a few other options but sometimes they work and sometimes they don't and I am not sure why. Such as cin.get().
P.S.S I am also new to programming my own stuff so I have limited resources as far as what I know already and will appreciate any ways of how I might "improve" my program to make it work more fluently. I also take criticism well so please do.
#include "../../std_lib_facilities.h"
int main(){
vector<int>Fibonacci;
int one = 0;
int two = 1;
int three = 0;
int i = 0;
while (i < 4000000){
i += three;
three = two + one; one = two; two = three;
cout << three << ", ";
Fibonacci.push_back(three);
//all of the above is to produce the Fibonacci number sequence which starts with 1, 2 and adds the previous one to the next so on and so forth.
//bellow is my attempt and taking those numbers and testing for evenness or oddness and then adding the even ones together for one single number.
}
cout << endl;
//go through all points in the vector Fibonacci and execute code for each point
for (i = 0; i <= 31; ++i)
if (Fibonacci.at(i) % 2 == 0)//is Fibonacci.at(i) even?
cout << Fibonacci.at(i) << endl;//how to get these numbers to add up to one single sum
system("pause");
}
Just do it by hand. That is loop over the whole array and and keep track of the cumulative sum.
int accumulator = 0; // Careful, this might Overflow if `int` is not big enough.
for (i = 0; i <= 31; i ++) {
int fib = Fibonacci.at(i);
if(fib % 2)
continue;
cout << fib << endl;//how to get these numbers to add up to one single sum
accumulator += fib;
}
// now do what you want with "accumulator".
Be careful about this big methematical series, they can explode really fast. In your case I think the calulation will just about work with 32-bit integers. Best to use 64-bit or even better, a propery BigNum class.
In addition to the answer by Adrian Ratnapala, I want to encourage you to use algorithms where possible. This expresses your intent clearly and avoids subtle bugs introduced by mis-using iterators, indexing variables and what have you.
const auto addIfEven = [](int a, int b){ return (b % 2) ? a : a + b; };
const auto result = accumulate(begin(Fibonacci), end(Fibonacci), 0, addIfEven);
Note that I used a lambda which is a C++11 feature. Not all compilers support this yet, but most modern ones do. You can always define a function instead of a lambda and you don't have to create a temporary function pointer like addIfEven, you can also pass the lambda directly to the algorithm.
If you have trouble understanding any of this, don't worry, I just want to point you into the "right" direction. The other answers are fine as well, it's just the kind of code which gets hard to maintain once you work in a team or have a large codebase.
Not sure what you're after...
but
int sum=0; // or long or double...
for (i = 0; i <= 31; ++i)
if (Fibonacci.at(i) % 2 == 0) {//is Fibonacci.at(i) even?
cout << Fibonacci.at(i) << endl;//how to get these numbers to add up to one single sum
sum+=Fibonacci.at(i);
}
// whatever
}

randomly choosing an empty vector element, when it is possible to know beforehand which are full

I finally determined that this function is responsible for the majority of my bottleneck issues. I think its because of the massively excessive random access that happens when most of the synapses are already active. Basically, as the title says, I need to somehow optimize the algorithm so that I'm not randomly checking a ton of active elements before landing on one of the few that are left.
Also, I included the whole function in case of other flaws that can be spotted.
void NetClass::Explore(vector <synapse> & synapses, int & n_syns) //add new synapses
{
int size = synapses.size();
assert(n_syns <= size );
//Increase the age of each active synapse by 1
Age_Increment(synapses);
//make sure there is at least one inactive vector left
if(n_syns == size)
return;
//stochastically decide whether a new connection is added
if((rand_r(seedp) %1000) < ( x / (1 +(n_syns * ( y / 100)))))
{
n_syns++; //a new synapse has been created
//main inefficiency here
while(1)
{
int syn = rand_r(seedp) % (size);
if (!synapses[syn].active)
{
synapses[syn].active = true;
synapses[syn].weight = .04 + (float (rand_r(seedp) % 17) / 100);
break;
}
}
}
}
void NetClass::Age_Increment(vector <synapse> & synapses)
{
for(int q=0, int size = synapses.size(); q < size; q++)
if(synapses[q].active)
synapses[q].age++;
}
Pass a random number, k, in the range [0, size-n_syns) to Age_Increment. Have Age_Increment return the kth empty slot.
Since you're already traversing the whole list in Age_Increment, update that function to return the list of the indexes of inactive synapses.
You can then pick a random item from that list directly.
This is similar to the problem of finding free blocks in memory management, so I would take a look at algorithms used in that domain, specifically free lists, which is a list of free positions. (These are usually implemented as linked lists to be able to pop elements off an end efficiently. Random access in a linked list would still be O(n) - with a smaller n, but still not the best choice for your use case.)