Differences between slice and container/list - list

Question

I'm just getting starting with Go and I have a situation where I need to make a set of entities whose size/length is only known at runtime. I first thought using a list would be a good fit but soon realized slices are the idiomatic data structure in Go.
Curious, I wrote the following benchmarks
package main
import (
"container/list"
"testing"
)
var N = 10000000
func BenchmarkSlices(B *testing.B) {
s := make([]int, 1)
for i := 0; i < N; i += 1 {
s = append(s, i)
}
}
func BenchmarkLists(B *testing.B) {
l := list.New()
for i := 0; i < N; i += 1 {
l.PushBack(i)
}
}
which gave me
BenchmarkSlices-4 2000000000 0.03 ns/op
BenchmarkLists-4 1 1665489308 ns/op
Given that append would create a new array and copy all the data over from the old to the new array when the old array gets full, I was expected lists to perform better than slices in the example above. However, my expectations are obviously wrong and I'm trying to understand why.
I wrote the following in order to understand a little better how append creates new arrays when it needs to:
package main
import "fmt"
func describe(s []int) {
fmt.Printf("len = %d, cap = %d\n", len(s), cap(s))
}
func main() {
s := make([]int, 2)
for i := 0; i < 15; i += 1 {
fmt.Println(i)
describe(s)
s = append(s, i)
}
}
which gave me
0
len = 2, cap = 2
1
len = 3, cap = 4
2
len = 4, cap = 4
3
len = 5, cap = 8
4
len = 6, cap = 8
5
len = 7, cap = 8
6
len = 8, cap = 8
7
len = 9, cap = 16
8
len = 10, cap = 16
9
len = 11, cap = 16
10
len = 12, cap = 16
11
len = 13, cap = 16
12
len = 14, cap = 16
13
len = 15, cap = 16
14
len = 16, cap = 16
My only guess at the moment as for why slices perform better than lists is that allocating memory for a new array whose size is double and copying all the data over is faster than allocating memory for a single element each time an insertion happens.
Is my guess correct? Am I missing something?

Answer 1

You are running the benchmarks wrong. You should first set up the initial data structure and then run the operation being benchmarked as many times as the testing.B instance indicates.
I replaced your code with:
var N = 1
func BenchmarkSlices(B *testing.B) {
s := make([]int, 1)
for n := 0; n < B.N; n++ {
for i := 0; i < N; i++ {
s = append(s, i)
}
}
}
func BenchmarkLists(B *testing.B) {
l := list.New()
for n := 0; n < B.N; n++ {
for i := 0; i < N; i++ {
l.PushBack(i)
}
}
}
And got this result:
BenchmarkSlices-4 100000000 14.3 ns/op
BenchmarkLists-4 5000000 275 ns/op
At least this time the difference seems reasonable and not like a trillion times.
Note that I also replaced the value of N to 1 so that ns/op actually means nanoseconds per operation and not nanoseconds per N operations. However, this might also impact the results.
Now onto your question: linked lists, as implemented in Go, suffer from additional costs when compared to simply adding another int to a pre-allocated slice: the list method needs to create a new Element, wrap your value in an interface{} and reassign some pointers.
Meanwhile appending to a slice which has not maxed out its capacity will result in just a few instructions at the CPU level: move an int to a memory location, increment the length of the slice, and you're done.
There is also the fact that the underlying allocator might reallocate the slice in place thus avoiding the need to copy the existing underlying array elements at all.