I'm getting memory limit exceeded error for this code. I can't find the way to resolve it. If I'm taking a long long int it gives the same error.
Why this error happening?
#include<bits/stdc++.h>
#define ll long long int
using namespace std;
int main()
{
///1000000000000 500000000001 getting memory limit exceed for this test case.
ll n,k;
cin>>n>>k;
vector<ll> v;
vector<ll> arrange;
for(ll i=0;i<n;i++)
{
v.push_back(i+1);
}
//Arranging vector like 1,3,5,...2,4,6,....
for(ll i=0;i<v.size();i++)
{
if(v[i]%2!=0)
{
arrange.push_back(v[i]);
}
}
for(ll i=0;i<v.size();i++)
{
if(v[i]%2==0)
{
arrange.push_back(v[i]);
}
}
cout<<arrange[k-1]<<endl; // Found the kth number.
return 0;
}
The provided code solves a coding problem for small values of n and k. However as you noticed it does fail for large values of n. This is because you are trying to allocate a couple of vectors of 1000000000000 elements, which exceeds the amount of memory available in today's computers.
Hence I'd suggest to return to the original problem you're solving, and try an approach that doesn't need to store all the intermediary values in memory. Since the given code works for small values of n and k, you can use the given code to check whether the approach without using vectors works.
I would suggest the following steps to redesign the approach to the coding problem:
Write down the contents of arrange for a small value of n
Write down the matching values of k for each element of arrange
Derive the (mathematical) function that maps k to the matching element in arrange
For this problem this can be done in constant time, so there is no need for loops
Note that this function should work both for even and odd values of k.
Test whether your code works by comparing it with your current results.
I would suggest trying the preceding steps first to come up with your own approach. If you can not find a working solution, please have a look at this approach on Wandbox.
Assume long long int is a 8 byte type. This is a commonly valid assumptions.
For every entry in the array, you are requesting to allocate 8 byte.
If you request to allocate 1000000000000 items, your are requesting to allocate 8 Terabyte of memory.
Moreover you are using two arrays and you are requesting to allocate more than 8 Terabyte.
Just use a lower number of items for your arrays and it will work.
Related
I am trying to solve Leetcode problem 746 which is a basic Dynamic programming problem. Even though my logic is little complex, it should work perfectly fine. I have had 2 sleepless nights trying to find what the problem in my code is. Can someone point exactly where am I doing heap overflow and how to avoid it ?
Also, I forgot to add, the size of cost is always greater than 3.
I have already tried inserting comments into my solution. I have realized that the problem lies with the costing[1] updating code but what the problem is, I have no idea.
class Solution {
public:
int minCostClimbingStairs(vector<int>& cost) {
if(cost.size() < 3)
return 0;
int costing[2];
costing[0]=cost[0];
costing[1]=cost[1];
int i=1;
while(i<cost.size()-2)
{
if(costing[0]+cost[i]>costing[0]+cost[i+1])
{
costing[0]=costing[0]+cost[i+1];
i++;
}
else if (costing[0]+cost[i]<=costing[0]+cost[i+1])
{
costing[0]=costing[0]+cost[i];
i+=2;
}
if(costing[1]+cost[i+1]>=costing[1]+cost[i+2])
{
cout<<costing[1]+cost[i+1]<<"is greater than " <<costing[1]+cost[i+2]<<"\n";
costing[1]+=cost[i+2];
i+=2;
}
else if (costing[1]+cost[i+1]<costing[1]+cost[i+2])
{
cout<<costing[1]+cost[i+1]<<"is less than " <<costing[1]+cost[i+2]<<"\n";
costing[1]+=cost[i+1];
i++;
}
}
return min(costing[0],costing[1]);
}
};
The initial value of i is 1. It can increment by 4 in one iteration of the while under different conditions (if the first else if and the second if are true).
So in the second iteration of the while, the value of i can become 9.
So in the last else if, cost[i+2] is cost[11]. Since your dataset has only 10 elements (as mentioned in the comment), this results in an out-of-bounds access.
Couldn't it be that your heap has not enough space? Vector allocate continuous memory blocks on the heap. If a vector grows for a long time, the heap may get fragmented. So at some point there is no free block available.
Maybe using std::vector::reserve can help you prevent this behaviour.
Also keep in mind that the indexed access ([i]) to vectors doesn't include range checking as opposed to the at function.
I encountered a problem that requires the program to count the number of points within an interval. This problem provides a large amount of unsorted points, and lo,hi(restriction lo<=hi), and it aims to enumerate the points within [lo,hi]. The problem is that although my code is correct, it is too time-consuming to finish within given time (2200ms). My code can finish this mission in O(n). I would like to ask if there are any faster methods.
int n,m,c,lo,hi;
cin>>n>>m;
int arr[n];
for(int i=0;i<n;i++){
cin>>arr[i];
}
cin>>lo>>hi;
c=0;
for(int j=0;j<n;j++){
if(arr[j]<=hi&&lo<=arr[j])c++;
}
cout<<c<<endl;
It is impossible to solve this problem in less than O(n) time, because you must consider all inputs at least once.
However, you might be able to reduce the constant factor of n — have you consider storing a set of (start, end) intervals, rather than a simple array? What is the input size which causes this to be slow?
Edit: upon further testing, it seems the bottleneck is actually the use of cin to read numbers.
Try replacing every instance of cin >> x; with scanf("%d", &x); — for me, this brings the runtime down to about 0.08 seconds.
You can do it faster than O(N) only if you need to do lookups more than once on the same data set:
Sort the array or its copy. For lookup you can use binary search - which is O(log2 N) complex.
Instead of flat array to use something like binary tree, lookup complexity will be as in #1.
I am trying to write a function in C++ using MPFR to calculate multiple values. I am currently using an mpfr array to store those values. It is unknown how many values need to be calculated and stored each time. Here is the function:
void Calculator(mpfr_t x, int v, mpfr_t *Values, int numOfTerms, int mpfr_bits) {
for (int i = 0; i < numOfTerms; i++) {
mpfr_init2(Values[i], mpfr_bits);
mpfr_set(Values[i], x, GMP_RNDN);
mpfr_div_si(Values[i], Values[i], pow(-1,i+1)*(i+1)*pow(v,i+1), GMP_RNDN);
}
}
The program itself has a while loop that has a nested for loop that takes these values and does calculations with them. In this way, I don't have to recalculate these values each time within the for loop. When the for loop is finished, I clear the memory with
delete[] Values;
before the the while loops starts again in which case, it redeclares the array with
mpfr_t *Values;
Values = new mpfr_t[numOfTerms];
The number of values that need to be stored are calculated by a different function and is told to the function through the variable numOfTerms. The problem is that for some reason, the array slows down the program tremendously. I am working with very large numbers so the thought is that if I recalculate those values each time, it gets extremely expensive but this method is significantly slower than just recalculating the values in each iteration of the for loop. Is there an alternative method to this?
EDIT** Instead of redeclaring the array over each time, I moved the declaration and the delete[] Values outside of the while loop. Now I am just clearing each element of the array with
for (int i = 0; i < numOfTerms; i++) {
mpfr_clear(Values[i]);
}
inside of the while loop before the while loop starts over. The program has gotten noticeably faster but is still much slower than just calculating each value over.
If I understand correctly, you are doing inside a while loop: mpfr_init2 (at the beginning of the iteration) and mpfr_clear (at the end of the iteration) on numOfTerms MPFR numbers, and the value of numOfTerms depends on the iteration. And this is what takes most of the time.
To avoid these many memory allocations by mpfr_init2 and deallocations by mpfr_clear, I suggest that you declare the array outside the while loop and initially call the mpfr_init2 outside the while loop. The length of the array (i.e. the number of terms) should be what you think is the maximum number of terms. What can happen is that for some iterations, the chosen number of terms was too small. In such a case, you need to increase the length of the array (this will need a reallocation) and call mpfr_init2 on the new elements. This will be the new length of the array for the remaining iterations, until the array needs to be enlarged again. After the while loop, do the mpfr_clear's.
When you need to enlarge the array, have a good strategy to choose the new number of elements. Just taking the needed value of numOfTerms for the current iteration may not be a good one, since it may yield many reallocations. For instance, make sure that you have at least a N% increase. Do some tests to choose the best value for N... See Dynamic array for instance. In particular, you may want to use the C++ implementation of dynamic arrays, as mentioned on this Wikipedia article.
TL;DR: Help me pass vector array input into a function that originally took in int pointers and ints.
I am in a college class that is learning about well-known algorithms. In this class, we use C++ to code the given algorithms. There was no pre-req class to learn C++, so my knowledge is pretty low when it comes to some of the major stuff with programming.
My problem: I have to create a program that takes an input file, sorts it with the user's choice of sorting algorithm, and write the results to an output file. My original code that works perfectly uses an input file of 20 items, placed into an array of length 20, and sorts no problem with each individual sorting algorithm.
Since last night, the only thing I have changed is that my input goes to a vector array, since the teacher will give us files of varying length (10 items to 1,000,000 items). I have four sorting algorithms that need to sort these given input files. Only one of them works, and it does not pass any variables into the function.
The other 3 originally passed in array pointers and other variables, however they do not work with my new input now going to a vector array instead of an int array. I know that what I am passing in needs to be changed, but I have no idea how to do this correctly. I have tried many different ways from sources found on the internet (with pointers and references), but I have had no luck. Here is some snipets of the code I'm using.
vector<int> A;
void insertionSort() // This works no problem as is.
void split(int* A, int* B, int low, int high) //code for Merge-Sort
{
//recurisvely calls split() and splitMerge()
}
void splitMerge(int* A, int* B, int low, int mid, int high) // more code for Merge-Sort
{
// more code for Merge-Sort
}
//quickSort() and countSort() also pass ints and do not work either.
//small part of main()
for (i = 0; unsorted >> temp; i++)
{
A.push_back(temp);
cout << A[i] << "\n";
length++; //I use int length extensively in for-loops in my functions
}
Last thing. I do not get an error when trying to run the Merge-Sort. It just freezes my console window.
conversion from vector to array is done this way
vector vInts;
...fill the vector
int * intArr=vInts[0]
so you don't need to modify your code too much
I believe there is not enough code to make an accurate prediction on where the error may be, but I think the problem is that the sorting algorithms is doing something like A++ with you pointers to access the next member.
Because arrays store the pointer to the next array inside the object and not on adjacent cells of memory, your algorithms are cycling through stuff they shouldn't.
If this is the case, the solution to your problem is to use an iterator instead of a pointer.
void split(A.begin(), B.begin(), int low, int high)
I was trying this problem on spoj.
First of all I came up with a sort of trivial o(blogb) algorithm(refer the problem for whats b).But since the author of the problem mentioned the constraints as b belongs to [0,10^7] i was not convinced if it would pass.Anyways out of shear belief I coded it as follows
#include<stdio.h>
#include<iostream>
#include<algorithm>
#include<cmath>
#include<vector>
#include<cstdlib>
#include<stack>
#include<queue>
#include<string>
#include<cstring>
#define PR(x) cout<<#x"="<<x<<endl
#define READ2(x,y) scanf("%d %d",&x,&y)
#define REP(i,a) for(long long i=0;i<a;i++)
#define READ(x) scanf("%d",&x)
#define PRARR(x,n) for(long long i=0;i<n;i++)printf(#x"[%d]=\t%d\n",i,x[i])
using namespace std;
#include <stdio.h>
struct node {
int val;
int idx;
};
bool operator<(node a,node b){ return a.val<b.val;}
node contain[10000001];
int main(){
int mx=1,count=1,t,n;
scanf("%d",&t);
while(t--){
count=1;mx=1;
scanf("%d",&n);
for(int i=0;i<n;i++){
scanf("%d",&contain[i].val);
contain[i].idx=i;
}
sort(contain,contain+n);
for(int j=1;j<n;j++){
if(contain[j].idx>contain[j-1].idx)
count++;
else count=1;
mx=max(count,mx);
}
printf("%d\n",n-mx);
}
}
And it passed in 0.01 s on SPOJ server(which is known to be slow)
But I soon came up with an O(b) algorithm,code given below
#include<stdio.h>
#include<iostream>
#include<algorithm>
#include<cmath>
#include<vector>
#include<cstdlib>
#include<stack>
#include<queue>
#include<string>
#include<cstring>
#define PR(x) printf(#x"=%d\n",x)
#define READ2(x,y) scanf("%d %d",&x,&y)
#define REP(i,a) for(int i=0;i<a;i++)
#define READ(x) scanf("%d",&x)
#define PRARR(x,n) for(int i=0;i<n;i++)printf(#x"[%d]=\t%d\n",i,x[i])
using namespace std;
int val[1001];
int arr[1001];
int main() {
int t;
int n;
scanf("%d",&t);
while(t--){
scanf("%d",&n);
int mn=2<<29,count=1,mx=1;
for(int i=0;i<n;i++){
scanf("%d",&arr[i]);
if(arr[i]<mn) { mn=arr[i];}
}
for(int i=0;i<n;i++){
val[arr[i]-mn]=i;
}
for(int i=1;i<n;i++){
if(val[i]>val[i-1]) count++;
else {
count=1;
}
if(mx<count) mx=count;
}
printf("%d\n",n-mx);
}
}
But surprisingly it took 0.14s :O
Now my question is isn't o(b) better than o(blogb) for b > 2 ? Then why so much difference in time? One of the members from the community suggested that it may be due to cache miss.The o(b) code is less localized as compared to o(blogb).But I dont see that causing a difference of 0.10s that too for <1000 runs of the code? (Yes b is actually less than 1000.Dont know why problem setter exaggerated so much)
EDIT : I see all answers are going towards the hidden constant values in asymptotic notations that often cause disparity in the running times of algorithms.But if you look at the codes you will realize all I am doing is replacing the call to sort by another traversal of the loop.Now I am assuming sort accesses each element of the array atleast once .Wouldn't that make both programs even closer if we think in number of lines that get executed?Beside yes my past experiences with spoj tells me I/O makes drastic impact on the running time of the program but I am using the same I/O routines in both the codes.
Big O notation describes how long the function takes as the input set approaches infinite size. If you have large enough data sets, O(n) will always beat O(n log n).
In practice, some 'poorer-performing' algorithms are faster because of the other hidden variables in the big O formula. Some more scalable algorithms can be slower. The difference becomes more arbitrary as the input set becomes smaller.
I learned all this the hard way, when I spent hours implementing a scalable solution, and when testing, found that it would only be faster for large data sets.
Edit:
Regarding the specific case, some people mentioned that the same line of code can vary extremely with regards to performance. This is likely the case here. That means that the 'hidden variables' in the big O formula are very relevant. The better you understand how a computer works on the inside, the more optimization techniques you have up your sleeve.
If you only remember one thing, remember this. Never compare two algorithms' performance by just reading the code. If it's that important, time an actual implementation on realistic data sets.
I/O operations (scanf(), printf()) are biasing the result.
those operations are notoriously slow and show a great discrepency when timing them. you shall never measure the performance of code using including any i/o operations, unless those operations are what you are trying to measure.
so, remove those calls and try again.
i will also point out that 0.1s is very small. the 0.1s difference may refer to the time it takes for loading the executable and preparing the code for execution.
Big-O notation isn't a formula that you can plug arbitrary values of n into. It merely describes the growth of the function as n heads to infinity.
This is a more interesting question than one might suspect. The O() concept can be useful, but it is not always as useful as some think. This is particularly true for logarithmic orders. Algebraically, a logarithm really has an order of zero, which is to say that log(n)/n^epsilon converges for any positive epsilon.
More often than we like to think, the log factors in order calculations don't really matter.
However, Kendall Frey is right. For sufficiently large data sets, O(n*log(n)) will eventually lose. It's only that the data set may have to be very large for the logarithmic difference to show.
I looked at your solutions in SPOj. I noticed that your O(nlogn) solution takes 79M memory while O(n) takes very small amount of memory which it shows as 0K. I looked at the other solutions too. Most of the fastest solutions I looked at used a large amount of memory. Now the most obvious reason I can think of is the implementation of std::sort() function. Its very nicely implemented which makes your solution amazingly fast. And for the O(n) solution i think it may be slow because of if() {...} else {...}. Try changing it to ternary operators and let us know if it makes any difference.
Hope it helps !!