Python - IndexError cannot fit 'long' into an index-sized integer - python-2.7

My code is working for small test cases but this is showing
IndexError: cannot fit 'long' into an index-sized integer
when I am using large numbers.
global mod
mod = 1000000007
global lim
lim = 10010
global cnt
cnt = [[None for x in range(0,4)] for x in range(0,lim+1)]
def precomp():
cnt[2][0] = cnt[2][1] = cnt[2][2] = 1
for i in range(3,lim):
cnt[i][0] = cnt[i-1][2] % mod
cnt[i][1] = cnt[i-1][0] % mod
cnt[i][2] = (cnt[i-1][2] + cnt[i-1][1]) % mod
t = input()
precomp()
for iterate in range( t ):
n = long(raw_input())
print long((cnt[n][0] + cnt[n][1] + cnt[n][2]) % mod)
When I test case with following input:
2
2
3
But when I am using following input:
3
28488106923106212
18451226948892907
5082944357138639
It is showing an error. What are the other ways I can set size of the 2D list or can overcome this error? By changing the range from(0,4) not giving the error but it is taking very long time.

Related

ValueError: invalid literal for int() with base 10 and could not convert string to float

I am trying to create if else odd and even code in python. I am getting 2 errors pls refer below.
n = int(raw_input())
if n%2 == 1:
The above code is raising error
invalid literal for int() with base 10: 'if n % 2 == 1:'
I modified the code to use float
n = float(raw_input())
if n % 2 = 1 :
ValueError: could not convert string to float: if n % 2 = 1 :
Please help on this issue
Use this:
if n%2==0:
code(when number is even)
else:
code(when number is odd)

Fibonacci sequence multiplying

I tried to make the fibonacci sequence with the following code:
def fibonacci(n): # write Fibonacci series up to n
"""Print a Fibonacci series up to n."""
a = 0
b = 1
the_list = []
while n > len(the_list):
the_list.append(a)
#By saying a = b and b = a+b we define the
#fibonacci sequence, since this is how the
#fibonacci sequence works.
a = b
b = a+b
print the_list
# Now call the function we just defined:
fibonacci(10)
As far as I know this code should do it but instead of giving me the fibonacci sequence its giving the following output:
[0, 1, 2, 4, 8, 16, 32, 64, 128, 256]
So my fibonacci sequence is multiplying instead of working correcly. I have no idea why because i thought
a = b
b = a+b
should do the trick, if i look at my while loop the statements for this loop are also correct, so I just dont get it why i dont get the right output.
So if someone could explain me why this code is not working it would be highly appriciated
Your code is creating an exponential sequence because of a logic flaw. Based on your code:
Start:
a = 0
b = 1
1st iteration:
a = b = 1
b = a + 1 = 1 + 1 = 2
2nd iteration:
a = b = 2
b = a + 2 = 2 + 2 = 4
As you can see the fact that you set a before performing the b calculation causes your issue.
Instead you need would something like (to prove the point):
tmp = a
a = b
b = tmp + a
A little extra math would eliminate the need for the extra variable:
b += a
a = b - a
But the easiest (and most pythonic) way would be:
a, b = b, a + b

Project Euler #2 in "Python"

I am an absolute beginner here. I was giving the questions on Project Euler a try in Python. Can you please point out where does my code go wrong?
Q) Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be:
1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...
By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.
def fib(a):
if ((a==0) or (a==1)):
return 1
else:
return((fib(a-1))+(fib(a-2)))
r=0
sum=0
while (fib(r))<4000000:
if(((fib(r))%2)==0):
sum+=fib(r)
print(sum)
Your code isn't wrong, it's just too slow. In order to solve Project Euler problems, not only does your code have to be correct, but your algorithm must be efficient.
Your fibonacci computation is extremely expensive - that is, recursively trying to attain the next fibonacci number runs in O(2^n) time - far too long when you want to sum numbers with a limit of four million.
A more efficient implementation in Python is as follows:
x = 1
y = 1
z = 0
result = 0
while z < 4000000:
z = (x+y)
if z%2 == 0:
result = result + z
#next iteration
x = y
y = z
print result
this definetly is not the only way- but another way of doing it.
def fib(number):
series = [1,1]
lastnum = (series[len(series)-1]+series[len(series)-2])
_sum = 0
while lastnum < number:
if lastnum % 2 == 0:
_sum += lastnum
series.append(lastnum)
lastnum = (series[len(series)-1] +series[len(series)-2])
return series,_sum
You should use generator function, here's the gist:
def fib(max):
a, b = 0, 1
while a < max:
yield a
a,b = b, a+b
Now call this function from the shell, or write a function after this calling the fib function, your problem will get resolved.It took me 7 months to solve this problem
This is probably the the most efficient way to do it.
a, b = 1, 1
total = 0
while a <= 4000000:
if a % 2 == 0:
total += a
a, b = b, a+b
print (total)
Using recursion might work for smaller numbers, but since you're testing every case up to 4000000, you might want to store the values that you've already found into values. You can look for this algorithm in existing answers.
Another way to do this is to use Binet's formula. This formula will always return the nth Fibonacci number. You can read more about this on MathWorld.
Note that even numbered Fibonacci numbers occur every three elements in the sequence. You can use:
def binet(n):
""" Gets the nth Fibonacci number using Binet's formula """
return int((1/sqrt(5))*(pow(((1+sqrt(5))/2),n)-pow(((1-sqrt(5))/2),n)));
s = 0; # this is the sum
i = 3;
while binet(i)<=4000000:
s += binet(i);
i += 3; # increment by 3 gives only even-numbered values
print(s);
You may try this dynamic program too, worked faster for me
dict = {}
def fib(x):
if x in dict:
return dict[x]
if x==1:
f = 1
elif x==2:
f = 2
else:
f = fib(x-1) + fib(x-2)
dict[x]=f
return f
i = 1
su = 0
fin = 1
while fin < 4000000:
fin = fib(i)
if fin%2 == 0:
su += fib(i)
i+=1
print (su)
As pointed in other answers your code lacks efficiency. Sometimes,keeping it as simple as possible is the key to a good program. Here is what worked for me:
x=0
y=1
nextterm=0
ans=0
while(nextterm<4000000):
nextterm=x+y
x=y
y=nextterm
if(nextterm%2==0):
ans +=nextterm;
print(ans)
Hope this helps. cheers!
it is optimized and works
def fib(n):
a, b = 0, 1
while a < n:
print(a, end=' ')
a, b = b, a+b
print()
fib(10000)
This is the slightly more efficient algorithm based on Lutz Lehmann's comment to this answer (and also applies to the accepted answer):
def even_fibonacci_sum(cutoff=4e6):
first_even, second_even = 2, 8
even_sum = first_even + second_even
while even_sum < cutoff:
even_fib = ((4 * second_even) + first_even)
even_sum += even_fib
first_even, second_even = second_even, even_fib
return even_sum
Consider the below Fibonacci sequence:
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, ...
Every third element in the Fibonacci sequence is even.
So the even numbers in the above sequence are 2, 8, 34, 144, 610, ...
For even number n, the below equation holds:
n = 4 * (n-1) + (n-2)
Example:
34 = (4 * 8) + 2, i.e., third even = (4 * second even) + first even
144 = (4 * 34) + 8, i.e., fourth even = (4 * third even) + second even
610 = (4 * 144) + 34 i.e., fifth even = (4 * fourth even) + third even
İt's can work with If we know in how many steps we will reach 4000000. It's around 30 steps.
a=1
b=2
list=[a,b]
for i in range (0,30):
a,b=b,a+b
if b%2==0:
list.append(b)
print(sum(list)-1)
Adapting jackson-jones answer to find the sum of the even-valued fibonacci terms below 4 million.
# create a function to list fibonacci numbers < n value
def fib(n):
a, b = 1, 2
while a < n:
yield a
a, b = b, a+b
# Using filter(), we extract even values from our fibonacci function
# Then we sum() the even fibonacci values that filter() returns
print(sum(filter(lambda x: x % 2 == 0, fib(4000000))))
The result is 4613732.

Does opening a file related to the program also stop the program?

I have this program that is supposed to search for perfect numbers.
(X is a perfect number if the sum of all numbers that divide X, divided by 2 is equal to X)
sum/2 = x
Now It has found the first four, which were known in Ancient Greece, so it's not really a anything awesome.
The next one should be 33550336.
I know it is a big number, but the program has been going for about 50 minutes, and still hasn't found 33550336.
Is it because I opened the .txt file where I store all the perfect numbers while the program was running, or is it because I don't have a PC fast enough to run it*, or because I'm using Python?
*NOTE: This same PC factorized 500 000 in 10 minutes (while also running the perfect number program and Google Chrome with 3 YouTube tabs), also using Python.
Here is the code to the program:
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 0
for x in range(1, i+1):
if i%x == 0:
sum += x
if sum / 2 == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
The next one should be 33550336.
Your code (I fixed the indentation so that it does in principle what you want):
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 0
for x in range(1, i+1):
if i%x == 0:
sum += x
if sum / 2 == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
does i divisions to find the divisors of i.
So to find the perfect numbers up to n, it does
2 + 3 + 4 + ... + (n-1) + n = n*(n+1)/2 - 1
divisions in the for loop.
Now, for n = 33550336, that would be
Prelude> 33550336 * (33550336 + 1) `quot` 2 - 1
562812539631615
roughly 5.6 * 1014 divisions.
Assuming your CPU could do 109 divisions per second (it most likely can't, 108 is a better estimate in my experience, but even that is for machine ints in C), that would take about 560,000 seconds. One day has 86400 seconds, so that would be roughly six and a half days (more than two months with the 108 estimate).
Your algorithm is just too slow to reach that in reasonable time.
If you don't want to use number-theory (even perfect numbers have a very simple structure, and if there are any odd perfect numbers, those are necessarily huge), you can still do better by dividing only up to the square root to find the divisors,
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 1
root = int(i**0.5)
for x in range(2, root+1):
if i%x == 0:
sum += x + i/x
if i == root*root:
sum -= x # if i is a square, we have counted the square root twice
if sum == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
that only needs about 1.3 * 1011 divisions and should find the fifth perfect number in a couple of hours.
Without resorting to the explicit formula for even perfect numbers (2^(p-1) * (2^p - 1) for primes p such that 2^p - 1 is prime), you can speed it up somewhat by finding the prime factorisation of i and computing the divisor sum from that. That will make the test faster for all composite numbers, and much faster for most,
def factorisation(n):
facts = []
multiplicity = 0
while n%2 == 0:
multiplicity += 1
n = n // 2
if multiplicity > 0:
facts.append((2,multiplicity))
d = 3
while d*d <= n:
if n % d == 0:
multiplicity = 0
while n % d == 0:
multiplicity += 1
n = n // d
facts.append((d,multiplicity))
d += 2
if n > 1:
facts.append((n,1))
return facts
def divisorSum(n):
f = factorisation(n)
sum = 1
for (p,e) in f:
sum *= (p**(e+1) - 1)/(p-1)
return sum
def isPerfect(n):
return divisorSum(n) == 2*n
i = 2
count = 0
out = 10000
while count < 5:
if isPerfect(i):
print i
count += 1
if i == out:
print "At",i
out *= 5
i += 1
would take an estimated 40 minutes on my machine.
Not a bad estimate:
$ time python fastperf.py
6
28
496
8128
33550336
real 36m4.595s
user 36m2.001s
sys 0m0.453s
It is very hard to try and deduce why this has happened. I would suggest that you run your program either under a debugger and test several iteration manually to check if the code is really correct (I know you have already calculated 4 numbers but still). Alternatively it would be good to run your program under a python profiler just to see if it hasn't accidentally blocked on a lock or something.
It is possible, but not likely that this is an issue related to you opening the file while it is running. If it was an issue, there would have probably been some error message and/or program close/crash.
I would edit the program to write a log-type output to a file every so often. For example, everytime you have processed a target number that is an even multiple of 1-Million, write (open-append-close) the date-time and current-number and last-success-number to a log file.
You could then Type the file once in a while to measure progress.

Very large execution time differences for virtually same C++ and Python code

I was trying to write a solution for Problem 12 (Project Euler) in Python. The solution was just too slow, so I tried checking up other people's solution on the internet. I found this code written in C++ which does virtually the same exact thing as my python code, with just a few insignificant differences.
Python:
def find_number_of_divisiors(n):
if n == 1:
return 1
div = 2 # 1 and the number itself
for i in range(2, n/2 + 1):
if (n % i) == 0:
div += 1
return div
def tri_nums():
n = 1
t = 1
while 1:
yield t
n += 1
t += n
t = tri_nums()
m = 0
for n in t:
d = find_number_of_divisiors(n)
if m < d:
print n, ' has ', d, ' divisors.'
m = d
if m == 320:
exit(0)
C++:
#include <iostream>
int main(int argc, char *argv[])
{
unsigned int iteration = 1;
unsigned int triangle_number = 0;
unsigned int divisor_count = 0;
unsigned int current_max_divisor_count = 0;
while (true) {
triangle_number += iteration;
divisor_count = 0;
for (int x = 2; x <= triangle_number / 2; x ++) {
if (triangle_number % x == 0) {
divisor_count++;
}
}
if (divisor_count > current_max_divisor_count) {
current_max_divisor_count = divisor_count;
std::cout << triangle_number << " has " << divisor_count
<< " divisors." << std::endl;
}
if (divisor_count == 318) {
exit(0);
}
iteration++;
}
return 0;
}
The python code takes 1 minute and 25.83 seconds on my machine to execute. While the C++ code takes around 4.628 seconds. Its like 18x faster. I had expected the C++ code to be faster but not by this great margin and that too just for a simple solution which consists of just 2 loops and a bunch of increments and mods.
Although I would appreciate answers on how to solve this problem, the main question I want to ask is Why is C++ code so much faster? Am I using/doing something wrongly in python?
Replacing range with xrange:
After replacing range with xrange the python code takes around 1 minute 11.48 seconds to execute. (Around 1.2x faster)
This is exactly the kind of code where C++ is going to shine compared to Python: a single fairly tight loop doing arithmetic ops. (I'm going to ignore algorithmic speedups here, because your C++ code uses the same algorithm, and it seems you're explicitly not asking for that...)
C++ compiles this kind of code down to a relatively few number of instructions for the processor (and everything it does probably all fits in the super-fast levels of CPU cache), while Python has a lot of levels of indirection it's going through for each operation. For example, every time you increase a number it's checking that the number didn't just overflow and need to be moved into a bigger data type.
That said, all is not necessarily lost! This is also the kind of code that a just-in-time compiler system like PyPy will do well at, since once it's gone through the loop a few times it compiles the code to something similar to what the C++ code starts at. On my laptop:
$ time python2.7 euler.py >/dev/null
python euler.py 72.23s user 0.10s system 97% cpu 1:13.86 total
$ time pypy euler.py >/dev/null
pypy euler.py > /dev/null 13.21s user 0.03s system 99% cpu 13.251 total
$ clang++ -o euler euler.cpp && time ./euler >/dev/null
./euler > /dev/null 2.71s user 0.00s system 99% cpu 2.717 total
using the version of the Python code with xrange instead of range. Optimization levels don't make a difference for me with the C++ code, and neither does using GCC instead of Clang.
While we're at it, this is also a case where Cython can do very well, which compiles almost-Python code to C code that uses the Python APIs, but uses raw C when possible. If we change your code just a little bit by adding some type declarations, and removing the iterator since I don't know how to handle those efficiently in Cython, getting
cdef int find_number_of_divisiors(int n):
cdef int i, div
if n == 1:
return 1
div = 2 # 1 and the number itself
for i in xrange(2, n/2 + 1):
if (n % i) == 0:
div += 1
return div
cdef int m, n, t, d
m = 0
n = 1
t = 1
while True:
n += 1
t += n
d = find_number_of_divisiors(t)
if m < d:
print n, ' has ', d, ' divisors.'
m = d
if m == 320:
exit(0)
then on my laptop I get
$ time python -c 'import euler_cy' >/dev/null
python -c 'import euler_cy' > /dev/null 4.82s user 0.02s system 98% cpu 4.941 total
(within a factor of 2 of the C++ code).
Rewriting the divisor counting algorithm to use divisor function makes the run time reduces to less than 1 second. It is still possible to make it faster, but not really necessary.
This is to show that: before you do any optimization trick with the language features and compiler, you should check whether your algorithm is the bottleneck or not. The trick with compiler/interpreter is indeed quite powerful, as shown in Dougal's answer where the gap between Python and C++ is closed for the equivalent code. However, as you can see, the change in algorithm immediately give a huge performance boost and lower the run time to around the level of algorithmically inefficient C++ code (I didn't test the C++ version, but on my 6-year-old computer, the code below finishes running in ~0.6s).
The code below is written and tested with Python 3.2.3.
import math
def find_number_of_divisiors(n):
if n == 1:
return 1
num = 1
count = 1
div = 2
while (n % div == 0):
n //= div
count += 1
num *= count
div = 3
while (div <= pow(n, 0.5)):
count = 1
while n % div == 0:
n //= div
count += 1
num *= count
div += 2
if n > 1:
num *= 2
return num
Here's my own variant built on nhahtdh's factor-counting optimization plus my own prime factorization code:
def prime_factors(x):
def factor_this(x, factor):
factors = []
while x % factor == 0:
x /= factor
factors.append(factor)
return x, factors
x, factors = factor_this(x, 2)
x, f = factor_this(x, 3)
factors += f
i = 5
while i * i <= x:
for j in (2, 4):
x, f = factor_this(x, i)
factors += f
i += j
if x > 1:
factors.append(x)
return factors
def product(series):
from operator import mul
return reduce(mul, series, 1)
def factor_count(n):
from collections import Counter
c = Counter(prime_factors(n))
return product([cc + 1 for cc in c.values()])
def tri_nums():
n, t = 1, 1
while 1:
yield t
n += 1
t += n
if __name__ == '__main__':
m = 0
for n in tri_nums():
d = factor_count(n)
if m < d:
print n, ' has ', d, ' divisors.'
m = d
if m == 320:
break