Generating primes without blowing the stack - ocaml

I'm learning OCaml (so forgive my style) and am trying to write a function that generates a list of prime numbers up to some upper bound. I've managed to do this in several different ways, all of which work until you scale them to a relatively high upper bound.
How can I change these (any of them) so that the recursion doesn't fill up the stack? I thought my while loop version would achieve this, but apparently not!
Generator
let primes max =
let isPrime p x =
let hasDivisor a = (x mod a = 0) in
not (List.exists hasDivisor p) in
let rec generate p test =
if test < max then
let nextTest = test + 2 in
if isPrime p test then generate (test :: p) nextTest
else generate p nextTest
else p in
generate [5; 3; 2] 7;;
This has been my most successful solution insofar as it doesn't immediately overflow the stack when running primes 2000000;;. However it just sits there consuming CPU; I can only assume it will complete eventually! The following alternatives all have the stack overflow problem:
Recursive Sieve of Eratosthenes
let primes max =
let rec sieve found toTest =
let h = List.hd toTest
and t = List.tl toTest in
let newPrimes = h :: found
and doesntDivide x = (x mod h <> 0) in
let nonDivisors = List.filter doesntDivide t in
if nonDivisors = [] then newPrimes
else sieve newPrimes nonDivisors in
let rec range a b =
if a > b then []
else a :: range (a + 1) b in
let p = range 2 max in
sieve [] p;;
Recursive Sieve of Eratosthenes v2
let primes max =
let rec sieve toTest =
let h = List.hd toTest
and t = List.tl toTest in
let doesntDivide x = (x mod h <> 0) in
let nonDivisors = List.filter doesntDivide t in
if nonDivisors = [] then [h]
else (h :: sieve nonDivisors) in
let rec range a b =
if a > b then []
else a :: range (a + 1) b in
let p = range 2 max in
sieve p;;
While Loop Sieve of Eratosthenes
let primes max =
let rec range a b =
if a > b then []
else a :: range (a + 1) b in
let tail = ref (range 2 max)
and p = ref [] in
while !tail <> [] do
let h = List.hd !tail
and t = List.tl !tail in
let doesntDivide x = (x mod h <> 0) in
let newTail = ref (List.filter doesntDivide t) in
tail := !newTail;
p := h :: !p
done;
!p;;

The stack overflows occur because your range function is not tail recursive. One that works is, e.g.
let rec range store a b =
if a > b then store
else range (a :: store) (a + 1) b
in
let p = List.rev (range [] 2 max) in
With that definition, and a format line, gives
$ ocamlopt -o primes2 primes2.ml
$ ./primes2
2
3
5
7
11
13
17
19
23
29
31
37
41
43
47
53
59
...
Since you're learning, I'll give you some unsolicited style comments as well :)
Don't use hd and tl. Prefer pattern matching. Then the compiler can tell you the cases you missed. E.g.
let rec sieve found toTest =
let h = List.hd toTest
and t = List.tl toTest in
would be
let rec sieve found = function
| h :: t -> ...
| [] -> Error handling...
Don't use x = []. Use pattern patching.
match x with
| [] -> ...
| h::t -> ...
Use anonymous functions rather than short (i.e. <= 1 line) named single use functions:
let doesntDivide x = (x mod h <> 0) in
let nonDivisors = List.filter doesntDivide t in
let nonDivisors = List.filter (fun x -> (x mod h <> 0)) t in
Use imperative features sparingly.

Your algorithms that you claim are the Sieve of Eratosthenes actually are not; they use trial division instead of sieving, which is easy to spot by looking for a comparison of a remainder (the mod operator) to zero. Here's a simple implementation of the Sieve of Eratosthenes, in pseudocode instead of Ocaml because it's been a long time since I wrote Ocaml code:
function primes(n)
sieve := makeArray(2..n, True)
for p from 2 to n
if sieve[p]
output p
for i from p*p to n step p
sieve[i] := False
That can be optimized further, though for small limits like n = 2000000 there is little point in doing so; in any case, a sieve will be very much faster than the trial division that you are using. If you're interested in programming with prime numbers, I modestly recommend this essay at my blog.

Related

Generate a Real list from a Int list in SML

Hello every body im training some SMLs and im creating a code to get deviation of a int list . in the process of it , i need to get a Real list out of some numbers in a int list , which it doesnt let me get them. heres my code :
fun mean [] = 0.0
| mean (first::rest) =
let
fun sum [] = 0
| sum (x::xs) = x + sum xs
fun counter [] = 0
| counter (y::ys) = 1 + counter ys
in
Real.fromInt (sum (first::rest)) / Real.fromInt (counter (first::rest))
end;
fun deviation [] = 0.0
| deviation (first::rest) =
let
fun diff (x::xs) = (x - mean (x::xs)) :: diff xs;
in
diff (first , first::rest) + deviation rest
end;
the problem is here :
fun diff (x::xs) = (x - mean (x::xs) ) :: diff xs;
diff is a recursive function, but the base case is never defined. When you try to run diff on an empty list, you will get a pattern match error.
You also define diff to accept a list, but you call it with a tuple.
You define diff as returning a list, given that you are using ::, but then you use addition on the result of that function, which will not work.
Improving mean
You can simplify your sum and counter functions with folds.
fun mean [] = 0.0
| mean lst =
let
val sum = foldl op+ 0 lst
val counter = foldl (fn (_, c) => c + 1) 0 lst
in
Real.fromInt sum / Real.fromInt counter
end;
But this requires iterating the entire list twice, when both pieces of information can be ascertained at the same time.
fun sumLen(lst) =
foldl (fn (x, (sum, len)) => (sum+x, len+1)) (0, 0) lst
mean can now be implemented as:
fun mean(lst) =
let
val (sum, len) = sumLen(lst)
in
Real.fromInt sum / Real.fromInt len
end
Deviation
To get the differences from the mean for a list, you need only use map.
fun diffs(lst) =
let
val m = mean(lst)
in
map (fn x => Real.fromInt x - m) lst
end
Consider evaluating the following.
diffs [1, 2, 3, 4, 5, 6, 7, 8]
The result is:
[~3.5, ~2.5, ~1.5, ~0.5, 0.5, 1.5, 2.5, 3.5]
From there you can use map and Math.pow to square those differences, foldl to sum them, divide by the length of the list, and then Math.sqrt to get the standard deviation.

How to use list comprenhension in Ocaml

This code is in Haskell. How can i do the same thing in OCAML?
perfect n = [x | x<-[1..n], sum(f x) == x]
f x = [i | i<-[1..x-1], x `mod` i ==0]
While Jeffrey's answer is correct, using appropriate libraries (in this case, sequence), you can get something that is similar in terseness and semantics to the Haskell style:
module S = Sequence
let sum = S.fold (+) 0
let f x = S.filter (fun i -> x mod i = 0) S.(1 -- (x-1))
let perfect n = S.filter (fun x -> sum (f x) = x) S.(1 -- n)
You're using many (really nice) features of Haskell that don't exist in OCaml.
For list comprehensions, you can use List.filter.
For the notation [x .. y] you can use this range function:
let range a b =
let rec go accum i =
if i > b then List.rev accum else go (i :: accum) (i + 1)
in
go [] a
For sum you can use this:
let sum = List.fold_left (+) 0

Core's `List.init` in Pervasives?

I'm used to JaneStreet's Core library. Its List module has a neat init function:
List.init;;
- : int -> f:(int -> 'a) -> 'a list = <fun>
It allows you to create a list with using a custom function to initialize elements:
List.init 5 ~f:(Fn.id);;
- : int list = [0; 1; 2; 3; 4]
List.init 5 ~f:(Int.to_string);;
- : string list = ["0"; "1"; "2"; "3"; "4"]
However, this function doesn't seem to exist in Pervasives, which is sad. Am I missing something, or do I have to implement it myself? And if I do need to write it, how do I achieve this?
EDIT:
I have written an imperative version of init, but it doesn't feel right to have to resort to OCaml's imperative features in such a case. :(
let init n ~f =
let i = ref 0 in
let l = ref [] in
while !i < n do
l := (f !i) :: !l;
incr i;
done;
List.rev !l
;;
EDIT 2:
I've opened a pull request on OCaml's GitHub to have this feature included.
EDIT 3:
The feature was released in OCaml 4.06.
A recursive implementation is fairly straightforward. However, it is not tail-recursive, which means that you'll risk a stack overflow for large lists:
let init_list n ~f =
let rec init_list' i n f =
if i >= n then []
else (f i) :: (init_list' (i+1) n f)
in init_list' 0 n f
We can transform it into a tail-recursive version using the usual techniques:
let init_list n ~f =
let rec init_list' acc i n f =
if i >= n then acc
else init_list' ((f i) :: acc) (i+1) n f
in List.rev (init_list' [] 0 n f)
This uses an accumulator and also needs to reverse the intermediate result, as the list is constructed in reverse. Note that we could also use f (n-i-1) instead of f i to avoid reversing the list, but this may lead to unexpected behavior if f has side-effects.
An alternative and shorter solution is to simply use Array.init as a starting point:
let init_list n ~f = Array.(init n f |> to_list)
You can copy the code from JaneStreet and use it.
The code look's like (but not exactly the same) :
let init n ~f =
if n < 0 then raise (Invalid_argument "init");
let rec loop i accum =
if i = 0 then accum
else loop (i-1) (f (i-1) :: accum)
in
loop n []
;;
You can find the original code inside core_list0.ml from the package core_kernel.

building a list of ints in ocaml

I want to write a function that does builds a list between two ints, inclusive
rec myFunc x y would build a list with all the ints between x and y, including x and y
For the logic right now I have something like this:
let rec buildList i n = let x = i+1 in if i <= n then i::(buildList x n)
But this gives me an error "Expression has type 'a list but but an expression was expected of type unit.
I thought buildList is returning a list of ints, and i as an int, so the cons operator would be valid, but its saying it should be void?
Why does this happen, and how do I fix it?
If the condition is true, you return the list i::(buildList x n). If it's not true, what do you return ?
Add else [] to your function to return the empty list when the condition is not met.
When you don't have any else, the compiler supposes it is else () (hence the error message).
Your if is missing an else condition
I suggest that you use a tail recursive function:
let buildList x y =
let (x,y) = if x<y then (x,y) else (y,x) in
let rec aux cpt acc =
if cpt < x then acc
else aux (cpt-1) (cpt::acc)
in aux y []
First, make sure that you ordered your boundaries correctly (idiot-proof), and then construct the list thank to a local recursive function which takes an accumulator.
Two alternatives relying on batteries' package,
Using unfold, which purpose is to build list,
let range ~from:f ~until:u =
BatList.unfold f (function | n when n <= u -> Some (n, succ n) | _ -> None)
Using Enum, allowing to work with lazy datastructure,
# BatList.of_enum ## BatEnum.(1--9);;
- : int list = [1; 2; 3; 4; 5; 6; 7; 8; 9]
My suggestion, this respects the ordering of the arguments.
let rec iota n m =
let oper = if n < m then succ else pred in
if n = m then [n] else n :: iota (oper n) m
Edit:
The operator selection is inside the recursive part, it should better be outside like this:
let iota n m =
let oper = if n < m then succ else pred in
let rec f1 n m = if n = m then [n] else n :: f1 (oper n) m in
f1 n m
At more than 200000 elements I get a stack overflow (so here we are)
# iota 0 250000;;
Stack overflow during evaluation (looping recursion?).
Todo: tail recursion
let buildList i n =
let rec aux acc i =
if i <= n then
aux (i::acc) (i+1)
else (List.rev acc)
in
aux [] i
Test:
# buildList 1 3;;
- : int list = [1; 2; 3]
# buildList 2 1;;
- : int list = []
# buildList 0 250000;;
- : int list =
[0; 1; 2; 3; .... 296; 297; 298; ...]

A List processing problem in F#

I am trying to do problem 12 in Project Euler.
numDivisor64 is to calculate number of divisors.
I wrote this F# code:
let problem12 =
{1L..300000L} |> Seq.map (fun x->x*(x+1L)/2L) |> Seq.map numDivisor64 |> Seq.filter (fun x->x>500L)
The problem asks to find the number rather than its # of divisors. Besides writing this in a less compact way using loops or recursion, any beautiful method?
Another problem, I occasionally find that I need to convert a 32-bit int version of code to a 64-bit one by adding 'L' to all the numbers. Is there a way to avoid this? Anything like c++ template?
I first wrote
let numDivisor n =
let rec countd n d =
if n%d=0 then
let n2, cnt = countd (n/d) d
n2, cnt+1
else
n, 0
let rec collect n d =
if n < d then 1
elif n%d=0 then
let n2, cnt = countd n d
(cnt+1) * (collect n2 d)
else
collect n (d+1)
collect n 2
Later I found I need bigger integers:
let numDivisor64 n =
let rec countd n d =
if n%d=0L then
let n2, cnt = countd (n/d) d
n2, cnt+1L
else
n, 0L
let rec collect n d =
if n < d then 1L
elif n%d=0L then
let n2, cnt = countd n d
(cnt+1L) * (collect n2 d)
else
collect n (d+1L)
collect n 2L
I would rephrase the search for the first wanted number as follows:
start with an infinite stream of int64's
turn them into triangle numbers
find the first number that satisfies the condition (instead of mapping to the number of divisors, which is not what you want, you want the number itself).
code:
let problem12 =
Seq.initInfinite int64 //the same as Seq.initInfinite (fun n -> int64 n)
|> Seq.map (fun x -> x*(x+1L)/2L)
|> Seq.find (fun x -> numDivisor64 x > 500L)
Regarding the second problem: when I solve project Euler problems I usually use int64's by default, because of type inference restrictions.
It's possible to write a more generic version using the inline keyword. See for instance this thread over at hubFS.
EDIT: here's a more generic version, using the technique described in the above link:
The type signature of NumDivisorG becomes horrible, but should work for any data type that 'knows' *,+,1 and 0.
module NumericLiteralG =
let inline FromZero() = LanguagePrimitives.GenericZero
let inline FromOne() = LanguagePrimitives.GenericOne
let inline numDivisorG n =
let rec countd n d =
if n%d=0G then
let n2, cnt = countd (n/d) d
n2, cnt+1G
else
n, 0G
let rec collect n d =
if n < d then 1G
elif n%d=0G then
let n2, cnt = countd n d
(cnt+1G) * (collect n2 d)
else
collect n (d+1G)
collect n (1G+1G)
let problem12L =
Seq.initInfinite int64 //the same as Seq.initInfinite (fun n -> int64 n)
|> Seq.map (fun x -> x*(x+1L)/2L)
|> Seq.find (fun x -> numDivisorG x > 500L)
let problem12I =
Seq.initInfinite id //the same as Seq.initInfinite (fun n -> n)
|> Seq.map (fun x -> x*(x+1)/2)
|> Seq.find (fun x -> numDivisorG x > 500)
if you have the list of divisors you could write a function to calculate the lowest common multiple of them all (which should be the number in question).
in haskell this looks like
lcmAll = foldl1 lcm
in F# i think it would look like this
let rec lcmAll ( head :: tail ) =
Seq.fold lcm head tail
I'm not sure if F# has a builtin lcm.
The alternative to this is to carry the original number around through all the transformations by using a product type, or tuple.
let problem12 =
{1L..300000L} |> Seq.map (fun x->x*(x+1L)/2L) |> Seq.map (fun x->(x,numDivisor64 x)) |> Seq.filter (fun (x,y)->y>500L)
In regards to the 64 bit number issue, if you give your function an explicit type signature it could force F# to use 64-bit ints (provided the type signature is valid for the function definition). Again this sort of thing works in Haskell, I cannot confirm it does with F#. If you could double check that would be awesome.