My Quicksort code works for some values of N (size of list), but for big values (for example, N = 82031) the error returned by OCaml is:
Fatal error: exception Stack_overflow.
What am I doing wrong?
Should I create an iterative version due to the fact that OCaml does not support recursive functions for big values?
let rec append l1 l2 =
match l1 with
| [] -> l2
| x::xs -> x::(append xs l2)
let rec partition p l =
match l with
| [] -> ([],[])
| x::xs ->
let (cs,bs) = partition p xs in
if p < x then
(cs,x::bs)
else
(x::cs,bs)
let rec quicksort l =
match l with
| [] -> []
| x::xs ->
let (ys, zs) = partition x xs in
append (quicksort ys) (x :: (quicksort zs));;
The problem is that none of your recursive functions are tail-recursive.
Tail-recursivity means that no further actions should be done by the caller (see here). In that case, there is no need to keep the environment of the caller function and the stack is not filled with environments of recursive calls. A language like OCaml can compile that in an optimal way but for this you need to provide tail-recursive functions.
For example, your first function, append :
let rec append l1 l2 =
match l1 with
| [] -> l2
| x::xs -> x::(append xs l2)
As you can see, after append xs l2 has been called, the caller needs to execute x :: ... and this function end up by not being tail-recursive.
Another way of doing it in a tail-recursive way is this :
let append l1 l2 =
let rec aux l1 l2 =
match l1 with
| [] -> l2
| x::xs -> append xs (x :: l2)
in aux (List.rev l1) l2
But, actually, you can try to use List.rev_append knowing that this function will append l1 and l2 but l1 will be reversed (List.rev_append [1;2;3] [4;5;6] gives [3;2;1;4;5;6])
Try to transform your other functions in tail-recursive ones and see what it gives you.
Best to fix the underlying problem as noted above, but if you really need a big stack, set ulimit -s. See also:
https://stackoverflow.com/a/71375559/14055985
Related
I have some code written in OCaml where I am trying to make a function that takes a list and sorts it via merge sort.
let rec msort ls =
let rec split lst (l1,l2) = match lst with
| [] -> (l1,l2)
| h::t -> split t (l2,h::l1) in
let merge l1 l2 =
let rec mergemerge l1 l2 acc = match (l1,l2) with
| (_,[]) -> l1
| ([],_) -> l2
| [], [] -> acc
| (h1::t1,h2::t2) -> if h1 < h2 then mergemerge t1 l2 (h1 :: acc)
else mergemerge l1 t2 (h2 :: acc)
in List.rev (mergemerge l1 l2 [])
in
let (l1,l2) =
split ls ([],[]) in merge (msort l1) (msort l2);;
When I try to compile the code, it says "Stack overflow during evaluation (looping recursion?)." I am wondering how to change the body so that it does not infinitely recurse and wondering how and where I would add base cases to the body. Thanks!
Your function calls itself recursively no matter what the input list looks like. So this will cause infinite recursion.
As you say, you need to check for the base case.
The base case for this function is pretty easy to see: are there any input lists that are already sorted? Yes, the empty list and a list containing just one element are already sorted.
Add an if ... then ... else as the outermost expression of msort. It should test for the base case and return the obvious result in that case. In other cases it should do what it does now.
I have recently started coding in ocaml and this programming language is hell sensible when it comes to defining what I want a function to return.
I want to write a function that uses 2 lists as parameters (supposed to be in ascending order and with elements of type int) and returns a list that contains all the elements of the first 2 lists, also in ascending order.
Here is what I managed to get so far:
let inter l1 l2 =
let rec aux l1 l2 l3=
if List.hd l1<List.hd l2 then aux (List.tl l1) l2 (List.hd l1 :: l3)
else (if List.hd l1>List.hd l2 then aux l1 (List.tl l2) (List.hd l2::l3)
else (if l1 = [] then List.fold_left (fun x y -> y::x) l3 l2
else if l2=[] then List.fold_left (fun x y -> y::x) l3 l1
))
in List.rev (aux l1 l2 []);;
But when I compile it, it returns me this error message:
Error: This expression has type 'a list
but an expression was expected of type unit
When I call the function, it works just fine, but it works just as expected, but what bothers me is the error message. Any idea why it appears?
PS: I use Emacs - Tuareg Mode as a text editor and compiler.
Th if/else syntactic construct is an expression. The type of the whole expression is defined by the type of expressions returned by the branches. Obviously, they must be of the same type. If you didn't specify the else branch, then it is assumed that the omitted else branch is an expression of type unit, basically if c then e is a shorthand for if c then e else ().
The expression:
if l2=[] then List.fold_left (fun x y -> y::x) l3 l1
is actually a shorthand for:
if l2=[] then List.fold_left (fun x y -> y::x) l3 l1 else ()
So OCaml tries to unify List.fold_left (fun x y -> y::x) l3 l1 with (). It they definitely have different types. If you add an explicit else branch, then everything will typecheck (not sure about the correctness):
let inter l1 l2 =
let rec aux l1 l2 l3=
if List.hd l1<List.hd l2 then aux (List.tl l1) l2 (List.hd l1 :: l3)
else (if List.hd l1>List.hd l2 then aux l1 (List.tl l2) (List.hd l2::l3)
else (if l1 = [] then List.fold_left (fun x y -> y::x) l3 l2
else if l2=[] then List.fold_left (fun x y -> y::x) l3 l1 else []
))
in List.rev (aux l1 l2 []);;
Your frustration may be due to being comfortable with C-like programming languages. OCaml can be frustrating when trying to force a C programming style in it. Pattern matching is a very powerful feature of OCaml and would simplify your solution:
let rec inter l1 l2 =
match l1, l2 with
| [], _ -> l2
| _, [] -> l1
| (h1 :: t1), (h2 :: t2) ->
if h1 <= h2 then
h1 :: inter t1 l2
else
h2 :: inter l1 t2
The first pattern says "if l1 is the null list, then return l2". The second pattern says "if l2 is an empty list, return l1". When pattern three is tested, we know both lists are not empty so we can pattern match on their contents, so there's no need to use list.hd, et al. We use an if-statement to determine which head becomes the new head and which tails we use when recursing.
As you get more comfortable with OCaml idioms, these solutions will come to you naturally.
So this is one way to append two lists:
let rec append l1 l2 =
match l1 with
| h :: t -> h :: append t l2
| [] -> l2
But I am trying to write a tail-recursive version of append. (solve the problem before calling the recursive function).
This is my code so far, but when I try to add append in the first if statement the code becomes faulty for weird reasons.
let list1 = [1;2;3;4]
let list2 = [5;6;7;8]
let rec append lista listb =
match listb with
| h :: taillist -> if taillist != [] then
begin
lista # [h];
(* I cant put an append recursive call here because it causes error*)
end else
append lista taillist;
| [] -> lista;;
append list1 list2;;
The easiest way to transform a non tail-recursive list algorithm into a tail-recursive one, is to use an accumulator. Consider rewriting your code using a third list, that will accumulate the result. Use cons (i.e., ::) to prepend new elements to the third list, finally you will have a result of concatenation. Next, you need just to reverse it with List.rev et voila.
For the sake of completeness, there is a tail-recursive append:
let append l1 l2 =
let rec loop acc l1 l2 =
match l1, l2 with
| [], [] -> List.rev acc
| [], h :: t -> loop (h :: acc) [] t
| h :: t, l -> loop (h :: acc) t l
in
loop [] l1 l2
I would recommend to solve 99 problems to learn this idiom.
A couple of comments on your code:
It seems like cheating to define a list append function using #, since this is already a function that appends two lists :-)
Your code is written as if OCaml were an imperative language; i.e., you seem to expect the expression lista # [h] to modify the value of lista. But OCaml doesn't work that way. Lists in OCaml are immutable, and lista # [h] just calculates a new value without changing any previous values. You would need to pass this new value in your recursive call.
As #ivg says, the most straightforward way to solve your problem is using an accumulator, with a list reversal at the end. This is a common idiom in a language with immutable lists.
A version using constant stack space, implemented with a couple of standard functions (you'll get a tail-recursive solution after unfolding the definitions):
let append xs ys = List.rev_append (List.rev xs) ys
Incidentally, some OCaml libraries implement the append function in a pretty sophisticated way:
(1) see core_list0.ml in the Core_kernel library: search for "slow_append" and "count_append"
(2) or batList.mlv in the Batteries library.
An alternative tail-recursive solution (F#) leveraging continuations :
let concat x =
let rec concat f = function
| ([], x) -> f x
| (x1::x2, x3) -> concat (fun x4 -> f (x1::x4)) (x2, x3)
concat id x
I think the best way to go about it, like some have said would be to reverse the first list, then recursively add the head to the front of list2, but the top comment with code uses an accumulator, when you can get the same result without it by :: to the second list instead of an accumulator
let reverse list =
let rec reverse_helper acc list =
match list with
| [] -> acc
| h::t -> reverse_helper (h::acc) t in
reverse_helper [] lst;;
let append list1 list2 =
let rec append_helper list1_rev list2 =
match list1_rev with
| [] -> list2
| h :: t -> append_helper t (h::lst2) in
append_helper (reverse lst1) lst2;;
A possible answer to your question could be the following code :
let append list1 list2 =
let rec aux acc list1 list2 = match list1, list2 with
| [], [] -> List.rev(acc)
| head :: tail, [] -> aux (head :: acc) tail []
| [], head :: tail -> aux (head :: acc) [] tail
| head :: tail, head' :: tail' -> aux (head :: acc) tail (head' :: tail')
in aux [] list1 list2;
It's pretty similar to the code given by another one of the commenters on your post, but this one is more exhaustive, as I added a case for if list2 is empty from the beginning and list1 isn't
Here is a simpler solution:
let rec apptr l k =
let ln = List.rev l in
let rec app ln k acc = match ln with
| [] -> acc
| h::t -> app t k (h::acc) in
app ln k k
;;
let rec append (mylist: 'a list) (myotherlist : 'a list ): 'a list =
match mylist with
| [] -> myotherlist
| a :: rest -> a :: append rest myotherlist
Is there a library function to find List1 minus elements that appear in List2? I've been googling around and haven't found much.
It doesn't seem too trivial to write it myself. I've written a function to remove a specific element from a list but that's much more simple:
let rec difference l arg = match l with
| [] -> []
| x :: xs ->
if (x = arg) then difference xs arg
else x :: difference xs arg;;
Will this do?
let diff l1 l2 = List.filter (fun x -> not (List.mem x l2)) l1
What I ended up actually doing was just writing another function which would call the first one I posted
let rec difference l arg = match l with
| [] -> []
| x :: xs ->
if (x = arg) then difference xs arg
else x :: difference xs arg;;
let rec list_diff l1 l2 = match l2 with
| [] -> l1
| x :: xs -> list_diff (difference l1 x) xs;;
Although the solution I accepted is much more elegant
I wrote this F# function to partition a list up to a certain point and no further -- much like a cross between takeWhile and partition.
let partitionWhile c l =
let rec aux accl accr =
match accr with
| [] -> (accl, [])
| h::t ->
if c h then
aux (h::accl) t
else
(accl, accr)
aux [] l
The only problem is that the "taken" items are reversed:
> partitionWhile ((>=) 5) [1..10];;
val it : int list * int list = ([5; 4; 3; 2; 1], [6; 7; 8; 9; 10])
Other than resorting to calling rev, is there a way this function could be written that would have the first list be in the correct order?
Here's a continuation-based version. It's tail-recursive and returns the list in the original order.
let partitionWhileCps c l =
let rec aux f = function
| h::t when c h -> aux (fun (acc, l) -> f ((h::acc), l)) t
| l -> f ([], l)
aux id l
Here are some benchmarks to go along with the discussion following Brian's answer (and the accumulator version for reference):
let partitionWhileAcc c l =
let rec aux acc = function
| h::t when c h -> aux (h::acc) t
| l -> (List.rev acc, l)
aux [] l
let test =
let l = List.init 10000000 id
(fun f ->
let r = f ((>) 9999999) l
printfn "%A" r)
test partitionWhileCps // Real: 00:00:06.912, CPU: 00:00:07.347, GC gen0: 78, gen1: 65, gen2: 1
test partitionWhileAcc // Real: 00:00:03.755, CPU: 00:00:03.790, GC gen0: 52, gen1: 50, gen2: 1
Cps averaged ~7s, Acc ~4s. In short, continuations buy you nothing for this exercise.
I expect you can use continuations, but calling List.rev at the end is the best way to go.
I usually prefer Sequences over List as they are lazy and you got List.toSeq and Seq.toList functions to convert between them. Below is the implementation of your partitionWhile function using sequences.
let partitionWhile (c:'a -> bool) (l:'a list) =
let fromEnum (e:'a IEnumerator) =
seq { while e.MoveNext() do yield e.Current}
use e = (l |> List.toSeq).GetEnumerator()
(e |> fromEnum |> Seq.takeWhile c |> Seq.toList)
,(e |> fromEnum |> Seq.toList)
You can rewrite the function like this:
let partitionWhile c l =
let rec aux xs =
match xs with
| [] -> ([], [])
| h :: t ->
if c h then
let (good, bad) = aux t in
(h :: good, bad)
else
([], h :: t)
aux l
Yes, as Brian has noted it is no longer tail recursive, but it answers the question as stated. Incidentally, span in Haskell is implemented exactly the same way in Hugs:
span p [] = ([],[])
span p xs#(x:xs')
| p x = (x:ys, zs)
| otherwise = ([],xs)
where (ys,zs) = span p xs'
A good reason for preferring this version in Haskell is laziness: In the first version all the good elements are visited before the list is reversed. In the second version the first good element can be returned immediately.
I don't think I'm the only one to learn a lot from (struggling with) Daniel's CPS solution. In trying to figure it out, it helped me change several potentially (to the beginner) ambiguous list references, like so:
let partitionWhileCps cond l1 =
let rec aux f l2 =
match l2 with
| h::t when cond h -> aux (fun (acc, l3) -> f (h::acc, l3)) t
| l4 -> f ([], l4)
aux id l1
(Note that "[]" in the l4 match is the initial acc value.) I like this solution because it feels less kludgey not having to use List.rev, by drilling to the end of the first list and building the second list backwards. I think the other main way to avoid .rev would be to use tail recursion with a cons operation. Some languages optimize "tail recursion mod cons" in the same way as proper tail recursion (but Don Syme has said that this won't be coming to F#).
So this is not tail-recursive safe in F#, but it makes my answer an answer and avoids List.rev (this is ugly to have to access the two tuple elements and would be a more fitting parallel to the cps approach otherwise, I think, like if we only returned the first list):
let partitionWhileTrmc cond l1 =
let rec aux acc l2 =
match l2 with
| h::t when cond h -> ( h::fst(aux acc t), snd(aux acc t))
| l3 -> (acc, l3)
aux [] l1