I was debugging an error and found out that undefined had been appended to a list, which caused a crash later on.
I expected that appending something other than a list with the ++ operator would cause a crash. But this is not true for undefined. Here is an example:
1> [1,2,3] ++ undefined.
[1,2,3|undefined]
Although it does not crash, the list is not fully functional anymore:
1> L = [1,2,3] ++ undefined.
[1,2,3|undefined]
2> L ++ [4].
** exception error: bad argument
in operator ++/2
called as [1,2,3|undefined] ++ [4]
Why does this happen?
Is this related to the underlying implementation of lists in erlang?
The reason is the ++ appends it second argument to the end of its first argument which must be a list. It does not do any processing of its second argument, it just appends it as is. So:
1> [1,2,3] ++ undefined.
[1,2,3|undefined]
2> [1,2,3] ++ [undefined].
[1,2,3,undefined]
The reason you can do this as well as:
3> [a|b].
[a|b]
4> [a|[b]].
[a,b]
is that a list is a sequence list cells, a singly linked list, not a single data structure as such. If the right-hand-sides, called the tail, of each cell is another list cell or [] then you get a proper list. The left-hand-side of each cell is called the head and usually contains the elements of the list. This is what we have in 2 and 4 above. Most, if not all, library functions assume that lists are proper lists and will generate an error if they are not. Note that you have to actually step down the whole list to the end to see if it is proper or not.
Each list cell is written as [Head|Tail] and the syntax [a,b,c] is just syntactic sugar for [a|[b|[c|[]]]]. Note that the tail of each list cell is a list or [] so this is a proper list.
There are no restrictions as to what the types the head and tail of a list cell can be. The system never checks it just does it. This what we have in 1 and 3 above where the tail of the last list cell (only list cell in 3) is not a list or [].
Sorry getting a bit over-didactic here.
EDIT: I see that I have already described this here: Functional Programming: what is an "improper list"?
In Erlang, all terms are represented by a compact pointer-like value called Eterm. It appears that list manipulation functions are implemented as type-agnostic.
Consider from this perspective: inside the erlang VM all Eterms are equal. The head and tail list manipulation operations are billed as being very fast. Since it takes several operations to evaluate the opaque Eterm type to determine whether or not it is a list, why bother?
The expected outcome in such a situation is an error, and you do get one. Eventually.
There's something to be said for trusting the programmer, and when dealing with an operation that adds several cycles and is used frequently, the potential benefit of ignoring a bad append stacks up, and the only penalty is a strange error.
Related
I've recently started learning scala, and I've come across the :: (cons) function, which prepends to a list.
In the book "Programming in Scala" it states that there is no append function because appending to a list has performance o(n) whereas prepending has a performance of o(1)
Something just strikes me as wrong about that statement.
Isn't performance dependent on implementation? Isn't it possible to simply implement the list with both forward and backward links and store the first and last element in the container?
The second question I suppose is what I'm supposed to do when I have a list, say 1,2,3 and I want to add 4 to the end of it?
The key is that x :: somelist does not mutate somelist, but instead creates a new list, which contains x followed by all elements of somelist. This can be done in O(1) time because you only need to set somelist as the successor of x in the newly created, singly linked list.
If doubly linked lists were used instead, x would also have to be set as the predecessor of somelist's head, which would modify somelist. So if we want to be able to do :: in O(1) without modifying the original list, we can only use singly linked lists.
Regarding the second question: You can use ::: to concatenate a single-element list to the end of your list. This is an O(n) operation.
List(1,2,3) ::: List(4)
Other answers have given good explanations for this phenomenon. If you are appending many items to a list in a subroutine, or if you are creating a list by appending elements, a functional idiom is to build up the list in reverse order, cons'ing the items on the front of the list, then reverse it at the end. This gives you O(n) performance instead of O(n²).
Since the question was just updated, it's worth noting that things have changed here.
In today's Scala, you can simply use xs :+ x to append an item at the end of any sequential collection. (There is also x +: xs to prepend. The mnemonic for many of Scala's 2.8+ collection operations is that the colon goes next to the collection.)
This will be O(n) with the default linked implementation of List or Seq, but if you use Vector or IndexedSeq, this will be effectively constant time. Scala's Vector is probably Scala's most useful list-like collection—unlike Java's Vector which is mostly useless these days.
If you are working in Scala 2.8 or higher, the collections introduction is an absolute must read.
Prepending is faster because it only requires two operations:
Create the new list node
Have that new node point to the existing list
Appending requires more operations because you have to traverse to the end of the list since you only have a pointer to the head.
I've never programmed in Scala before, but you could try a List Buffer
Most functional languages prominently figure a singly-linked-list data structure, as it's a handy immutable collection type. When you say "list" in a functional language, that's typically what you mean (a singly-linked list, usually immutable). For such a type, append is O(n) whereas cons is O(1).
I have to write a Union function using recursion
The ouput has to be the Union (no duplicates) of two lists. My teacher said the implementation has to be recursive and we cannot go through the lists twice but I don't think I can come up with a way of solving the problem without going through the lists twice?
My ideas which would solve the problem (but involve going through lists twice):
- Merge then remove duplicates
- Sorting the lists, then merge
Any hints or help would be appreciated
Edit: Well so I got to combine both lists by doing this:
union1 :: (Eq a) => [a] -> [a] -> [a]
union1 xs [] = xs
union1 [] ys = ys
union1 (x:xs)(y:ys) = x:y:union1(xs)(ys)
Then I thought I could use nub or a similar function to remove the duplicates but I got stuck thinking because then I would be going through the lists twice, right?
What is list union?
I would like to first point out that the requirements your teacher gave you are a bit vague. Moreover, union on multisets (aka sets that can have duplicates, like lists) have two different definitions in mathematics (other source). I am no mathematician, but here is what I was able to glean from various internets. Here is one definition:
λ> [1,2,2,3,3,3] `unionA` [1,2,2,2,3] --also called multiset sum
[1,1,2,2,2,2,2,3,3,3,3]
This is simply (++), if you're not worried about ordering. And here is the other:
λ> [1,2,2,3,3,3] `unionB` [1,2,2,2,3]
[1,2,2,2,3,3,3] --picks out the max number of occurrences from each list
Adding to this confusion, Data.List implements a somewhat quirky third type of union, that treats its left input differently from its right input. Here is approximately the documentation found in comments the source code of union from Data.List:
The union function returns the list union of the two lists. Duplicates, and elements of the first list, are removed from the the second list, but if the first list contains duplicates, so will the result. For example,
λ> "dog" `union` "cow"
"dogcw"
Here, you have 3 possible meanings of "union of lists" to choose from. So unless you had example input and output, I don't know which one your teacher wants, but since the goal is probably for you to learn about recursion, read on...
Removing duplicates
Removing duplicates from unordered lists in Haskell can be done in linear time, but solutions involve either random-access data structures, like Arrays, or something called "productive stable unordered discriminators", like in the discrimination package. I don't think that's what your teacher is looking for, since the first is also common in imperative programming, and the second is too complex for a beginner Haskell course. What your teacher probably meant is that you should traverse each list explicitly only once.
So now for the actual code. Here I will show you how to implement union #3, starting from what you wrote, and still traverse the lists (explicitly) only once. You already have the basic recursion scheme, but you don't need to recurse on both lists, since we opted for option #3, and therefore return the first list unchanged.
Actual code
You'll see in the code below that the first list is used as an "accumulator": while recursing on the second list, we check each element for a duplicate in the first list, and if there isn't a duplicate, we append it to the first list.
union [] r = r
union l [] = l
unionR ls (r:rs)
| r `elem` ls = unionR ls rs --discard head of second list if already seen
--`elem` traverses its second argument,
--but see discussion above.
| otherwise = unionR (r:ls) rs --append head of second list
As a side note, you can make this a bit more readable by using a fold:
union xs = foldl step xs where --xs is our first list; no recursion on it,
--we use it right away as the accumulator.
step els x
| x `elem` els = els
| otherwise = x : els
I've been trying to figure out what list another list representation corresponds to:
[1|[2|[3|[4|[]]]]] is equivalent to the list [1,2,3,4] and [1,2,3,4|[]] etc.
But I just can't seem to figure out what list this corresponds to:
[[[[[]|1]|2]|3]|4]
If anyone can explain this for me I would be very grateful.
[H|T] is syntactic sugar for the real representation using the . functor: .(H,T) where H is the "head" (one list element), and T is the "tail" (which is itself a list in a standard list structure). So [1|[2|[3|[4|[]]]]] is .(1,.(2,.(3,.(4,[])))). Prolog also allows a non-list value for T, so [[[[[]|1]|2]|3]|4] is .(.(.(.([],1),2),3),4). The second structure doesn't really simplify any further than that from a list notational standpoint.
If you want to think in terms of "list" then [[[[[]|1]|2]|3]|4] is a list whose head is [[[[]|1]|2]|3] and tail is 4. And since 4 is not a list, the original list can be described as "improper" as #z5h indicated since the tail isn't a list (not even the empty list). Drilling down, the head [[[[]|1]|2]|3] is itself a list with head [[[]|1]|2] and tail 3. Another "improper" list. And so on. Therefore, the overall structure is an embedded list of lists, four levels deep, in which each list has a single head and a non-list tail (an "improper" list).
It's interesting to note that some of Prolog's predicates handle this type of list. For example:
append([], 1, L).
Will yield:
L = [[]|1]
You can then build your oddly formed list using append:
append([[]], 1, L1), % append 1 as the tail to [[]] giving L1
append([L1], 2, L2), % append 2 as the tail to [L1] giving L2
append([L2], 3, L3), % append 3 as the tail to [L2] giving L3
append([L3], 4, L4). % append 4 as the tail to [L3] giving L4
Which yields:
L4 = [[[[[]|1]|2]|3]|4]
Each append takes a list of one element (which is itself a list from the prior append, starting with [[]]) and appends a non-list tail to it.
It's called an "improper list".
Let me explain.
I think a few different ideas are at play here.
In general we know what a list is: it's an ordered collection.
How we represent a list literal in a programming language is another facet.
How the list is represented internally in the language, is another facet still.
What you have realized, is that there is a data structure you can represent in Prolog, with similar syntax to a list, but it somehow seems "improper". (It's not clear that you have asked much beyond "what is the meaning of this confusing thing?").
It turns out, that this structure is simply known as an "improper list". This data structure shows up often in languages that store lists internally as nested cons cells or similar. If you Google for that, you'll find plenty of resources for examples, usages, etc.
I have a list contains some elements and now with the help of lists:foreach I am fetching some more records and I want to append each value to my existing list elements without creating new variable as doing in other languages with help of array.
Here is my sample code which I am getting:
exception error: no match of right hand side value [6,7,1].
Sample Code:
listappend() ->
A = [1,2,3,4,5],
B = [6,7],
lists:foreach(fun (ListA) ->
B = lists:append(B, [ListA])
end, A),
B.
I want output like,
B = [6,7,1,2,3,4,5].
First of all, this feature already exists, so you won't need to implement it yourself. In fact, the list can take two lists as arguments:
1> lists:append([1,2,3,4,5], [6,7]).
[1,2,3,4,5,6,7]
Which is actually implemented as:
2> [1,2,3,4,5] ++ [6,7].
[1,2,3,4,5,6,7]
Please bear in mind that the ++ operator will copy the left operand, so this operation can easily lead to quadratic complexity. Said that, you probably want to construct your lists using the "cons" operator (eventually reversing the list at the end of the computation):
3> [1|[2,3,4,5,6,7]].
[1,2,3,4,5,6,7]
In any case, you can have two arguments in your function, which are the two lists to append, instead of defining them in the body of the function. This way, the values of A and B will change every time you call the my_append/2 function.
my_append(A, B) ->
YOUR_CODE_GOES_HERE
As a note and regarding the actual error you're getting, this is due to the following line:
B = lists:append(B, [ListA])
During each iteration, you're binding a new value to the variable B, which is already bound to the value [6,7].
Variables in Erlang are immutable. That means you cannot assign a new value to a variable once it has been bound, and this is why you get the 'no match' error. The old value does simply not match the new value you are trying to assign to the variable. Instead, you can create a new list using e.g lists:append based on the old one. You should probably start by looking at recursion and how you can use it to manipulate lists.
From what I understand, the list type in Haskell is implemented internally using a linked list. However, the user of the language does not get to see the details of the implementation, nor does he have the ability to modify the "links" that make up the linked list to allow it to point to a different memory address. This, I suppose, is done internally.
How then, can the list type be qualified as in Haskell ? Is it a "data type" or an "abstract data type"? And what of the linked list type of the implementation ?
Additionally, since the list type provided by the Prelude is not a linked list type, how can the basic linked list functions be implemented ?
Take, for example, this piece of code designed to add an element a at the index n of a list :
add [] acc _ _ = reverse acc
add (x:xs) acc 0 a = add xs (x:a:acc) (-1) a
add (x:xs) acc n a = add xs (x:acc) (n-1) a
Using a "real" linked list, adding an element would just consist of modifying a pointer to a memory address. This is not possible in Haskell (or is it ?), thus the question : is my implementation of adding an element to a list the best possible one, or am I missing something (the use of the reverse function is, I think, particularly ugly, but is it possible to do without ?)
Please, do not hesitate to correct me if anything I have said is wrong, and thank you for your time.
You're confusing mutability with data structure. It is a proper list — just not one you're allowed to modify. Haskell is purely functional, meaning values are constant — you can't change an item in a list any more than you could turn the number 2 into 3. Instead, you perform calculations to create new values with the changes you desire.
You could define that function most simply this way:
add ls idx el = take idx ls ++ el : drop idx ls
The list el : drop idx ls reuses the tail of the original list, so you only have to generate a new list up to idx (which is what the take function does). If you want to do it using explicit recursion, you could define it like so:
add ls 0 el = el : ls
add (x:xs) idx el
| idx < 0 = error "Negative index for add"
| otherwise = x : add xs (idx - 1) el
add [] _ el = [el]
This reuses the tail of the list in the same way (that's the el : ls in the first case).
Since you seem to be having trouble seeing how this is a linked list, let's be clear about what a linked list is: It's a data structure consisting of cells, where each cell has a value and a reference to the next item. In C, it might be defined as:
struct ListCell {
void *value; /* This is the head */
struct ListCell *next; /* This is the tail */
}
In Lisp, it's defined as (head . tail), where head is the value and tail is the reference to the next item.
In Haskell, it's defined as data [] a = [] | a : [a], where a is the value and [a] is the reference to the next item.
As you can see, these data structures are all equivalent. The only difference is that in C and Lisp, which are not purely functional, the head and tail values are things you can change. In Haskell, you can't change them.
Haskell is a purely functional programming language. This means no change can be done at all.
The lists are non-abstract types, it's just a linked list.
You can think of them defined in this way:
data [a] = a : [a] | []
which is exactly the way a linked list is defined - A head element and (a pointer to) the rest.
Note that this is not different internally - If you want to have more efficient types, use Sequence or Array. (But since no change is allowed, you don't need to actually copy lists in order to distinguish between copies so, which might be a performance gain as opposed to imperative languages)
In Haskell, "data type" and "abstract type" are terms of art:
A "data type" (which is not abstract) has visible value constructors which you can pattern-match on in case expressions or function definitions.
An "abstract type" does not have visible value constructors, so you cannot pattern match on values of the type.
Given a type a, [a] (list of a) is a data type because you can pattern match on the visible constructors cons (written :) and nil (written []). An example of an abstract type would be IO a, which you cannot deconstruct by pattern matching.
Your code might work, but it's definitely not optimal. Take the case where you want to insert an item at index 0. An example:
add [200, 300, 400] [] 0 100
If you follow the derivation for this, you end up with:
add [200, 300, 400] [] 0 100
add [300, 400] (200:100:[]) (-1) 100
add [400] (300:[200, 100]) (-2) 300
add [] (400:[300, 200, 100]) (-3) 400
reverse [400, 300, 200, 100]
[100, 200, 300, 400]
But we are only adding an item to the beginning of the list! Such an operation is simple! It's (:)
add [200, 300, 400] [] 0 100
100:[200, 300, 400]
[100, 200, 300, 400]
Think about how much of the list really needs to be reversed.
You ask about whether the runtime modifies the pointers in the linked list. Because lists in Haskell are immutable, nobody (not even the runtime) modifies the pointers in the linked list. This is why, for example, it is cheap to append an item to the front of a list, but expensive to append an element at the back of a list. When you append an item to the front of the list, you can re-use all of the existing list. But when you append an item at the end, it has to build a completely new linked list. The immutability of data is required in order for operations at the front of a list to be cheap.
Re: adding an element to the end of a List, I'd suggest using the (++) operator and splitAt function:
add xs a n = beg ++ (a : end)
where
(beg, end) = splitAt n xs
The List is a linked-list, but it's read-only. You can't modify a List in place - you instead create a new List structure which has the elements you want. I haven't read it, but this book probably gets at your underlying question.
HTH
The compiler is free to choose any internal representation it wants for a list. And in practice it does actually vary. Clearly the list "[1..]" is not implemented as a classical series of cons cells.
In fact a lazy list is stored as a thunk which evaluates to a cons cell containing the next value and the next thunk (a thunk is basically a function pointer plus the arguments for the function, which gets replaced by the actual value once the function is called). On the other hand if the strictness analyser in the compiler can prove that the entire list will always be evaluated then the compiler does just create the entire list as a series of cons cells.